Vim9: Crash when assigning to an element of an outer function's argument within a lambda - #21465

gdb backtrace and related variable values

Thread 1 "vim" received signal SIGSEGV, Segmentation fault.
exec_instructions (ectx=0x7ffd9e49b3e0) at vim9execute.c:4680
4680                            tv = ((typval_T *)outer->out_stack->ga_data)
(gdb) bt
#0  exec_instructions (ectx=0x7ffd9e49b3e0) at vim9execute.c:4680
#1  0x0000603b2169aa9a in call_def_function
    (ufunc=0x603b4717e860, argc_arg=0, argv=0x7ffd9e49be60, flags=0, partial=0x0, object=0x0, funccal=0x603b479286b0, rettv=0x7ffd9e49c830) at vim9execute.c:6999
#2  0x0000603b216611e5 in call_user_func
    (fp=0x603b4717e860, argcount=0, argvars=0x7ffd9e49be60, rettv=0x7ffd9e49c830, funcexe=0x7ffd9e49c030, selfdict=0x0) at userfunc.c:3092
#3  0x0000603b21662543 in call_user_func_check
    (fp=0x603b4717e860, argcount=0, argvars=0x7ffd9e49be60, rettv=0x7ffd9e49c830, funcexe=0x7ffd9e49c030, selfdict=0x0) at userfunc.c:3529
#4  0x0000603b216639a5 in call_func
    (funcname=0x603b45804d80 "Run()", len=3, rettv=0x7ffd9e49c830, argcount_in=0, argvars_in=0x7ffd9e49be60, funcexe=0x7ffd9e49c030) at userfunc.c:4202
#5  0x0000603b2165f3d9 in get_func_tv
    (name=0x603b45804d80 "Run()", len=3, rettv=0x7ffd9e49c830, arg=0x7ffd9e49c798, evalarg=0x7ffd9e49c840, funcexe=0x7ffd9e49c030) at userfunc.c:2194
#6  0x0000603b213f398c in eval_func
    (arg=0x7ffd9e49c798, evalarg=0x7ffd9e49c840, name=0x603b45c10320 "Run()", name_len=3, rettv=0x7ffd9e49c830, flags=1, basetv=0x0) at eval.c:3456
#7  0x0000603b213f78ab in eval9_var_func_name
    (arg=0x7ffd9e49c798, rettv=0x7ffd9e49c830, evalarg=0x7ffd9e49c840, evaluate=1, name_start=0x7ffd9e49c170) at eval.c:5365
...

(gdb) p iptr->isn_type
$2 = ISN_LOADOUTER

(gdb) p depth
$3 = 1
(gdb) p outer
$4 = (outer_T *) 0x576f40848118

(gdb) p outer->out_stack
$1 = (garray_T *) 0x0

Flow from the crash location to the cause

• Vim binary: 9.2.1167 with A large amount of ch_log is embedded.
• Log file: Gist The last 1243 lines of the log when Vim crashed with the above Vim binary
• Source code line numbers are those in the above branch
• Instruction numbers are the order in isntype_T in vim9.h (34=LOADOUTER, 88=FUNCREF, etc.)

1. Crash location
   SEGV at vim9execute.c:4680 (processing of ISN_LOADOUTER) because
   outer->out_stack is NULL. outer is
   ectx->ec_outer_ref->or_outer (vim9execute.c:4650).
   In the log, the last executed instruction is the lambda's LOADOUTER
   (34), immediately followed by Exiting... (1241, 1243).

2. Where does outer come from
   Set() in Run is executed by PCALL (83) (1238), and proceeds through
   call_partial() (vim9execute.c:1642)
   -> call_ufunc() (vim9execute.c:1693)
   -> call_dfunc() (vim9execute.c:1502)
   and at vim9execute.c:722 in call_dfunc() it becomes
   ref->or_outer = get_pt_outer(pt);. get_pt_outer() returns
   &pt->pt_outer (vim9execute.c:445).
   In other words, outer is the pt_outer itself of the partial
   stored in Set, and it means that pt_outer.out_stack has never
   been set.
   (This step was traced by reading the code; no log was taken)

3. Where is pt_outer.out_stack set
   3 places with grep -n "out_stack = " *.c. Of these, the one that
   initially sets the partial's pt_outer is only the third one,
   vim9execute.c:2124 in fill_partial_and_closure().
   However, it is set only when entering
   if (ufunc->uf_flags & FC_CLOSURE)
   at vim9execute.c:2117.
   (The remaining 2 places: vim9execute.c:735 is for the outer newly
   created when there is no partial, and vim9execute.c:927 is in
   handle_closure_in_use(), which re-points a partial already used as
   a closure to the funcstack)

4. No FC_CLOSURE in fill_partial_and_closure()
   It is called from FUNCREF (88) in MakeSetter (1231, 1232).
   fill_partial_and_closure(2113): in. ufunc->uf_name:"<lambda>1", sid:32, ufunc->uf_flags:0x00002400
   0x2400 = FC_LAMBDA (0x2000) | FC_VIM9 (0x400), and FC_CLOSURE
   (0x08) is absent (structs.h:2126-2137). Therefore it does not enter
   the if, and goes straight to out. OK (1233). Here a partial is
   created with out_stack still NULL, and it goes into Set of
   Run as the return value of MakeSetter.

5. Where is FC_CLOSURE set
   In compile_dfunc_epilogue() at the end of compilation, it is
   if (cctx->ctx_outer_used) (vim9compile.c:5076)
   ufunc->uf_flags |= FC_CLOSURE; (vim9compile.c:5078).
   In the log, the lambda's ctx_outer_used is 0 (1117, 1118).
   compile_dfunc_epilogue(5075): ctx_outer_used: 0
   Since the lambda uses the argument arg of the outer MakeSetter,
   it should originally be 1.
   (MakeSetter and Run are also 0 (1131, 1222), but neither has an
   outer function, so 0 is correct)

6. Why ctx_outer_used is not set
   ctx_outer_used is set to TRUE in only the following 2 places.
   vim9compile.c:117 When lookup_local() finds an outer local variable
   vim9expr.c:1129 When reading an outer variable or argument in an expression
   arg in the lambda body arg.key = 1 is the left-hand side of an
   assignment, so it goes through neither, and is looked up by
   arg_exists() at vim9compile.c:2026. In the log, lookup_local is
   FAIL both times (not among the local variables of the lambda itself
   or the outer MakeSetter)
   arg_exists(174): out. OK (in the recursive inner call, found as
   an argument of MakeSetter)
   arg_exists(201): out. OK (returned from the branch for when it
   was found in the outer)
   (1088-1095. 1096-1103 is the second time of the same lookup).
   This "found in the outer" branch (vim9compile.c:196-201)
   only increments gen_load_outer, and does not set ctx_outer_used.
   Even so, lv_from_outer becomes 1, so at vim9compile.c:1443
   LOADOUTER is generated (1108).

7. Summary
   When an argument of the outer function is used on the left-hand side
   of an assignment, arg_exists() does not set ctx_outer_used (6)
   -> FC_CLOSURE is not set on the lambda (5)
   -> pt_outer.out_stack is not set in fill_partial_and_closure() of FUNCREF (4)
   -> outer of the lambda called by Set() becomes that pt_outer (2)
   -> LOADOUTER references the NULL out_stack and causes a SEGV (1)

   The cause is the mismatch that LOADOUTER is generated, but the
   closure marker (FC_CLOSURE) is not set. For an outer local variable,
   lookup_local() sets it, and in an expression vim9expr.c:1129 sets it, so
   it occurs only with the combination of "left-hand side" and "outer argument".