Re: eval line number weirdness

["Rocky Bernstein" <rocky.bernstein@xxxxxxxxx>]

I applied the patch and it works great. Thanks.

I've fixed and committed my zshdb bugs in this area.  Now that the
line numbers are pretty accurate, I guess it's time to start adding
debugger breakpoints by file and line number. (Started tonight; The
corresponding bashdb code is is ugly and can be simplified and made
more reliable using associative arrays.)

Onward and upward. Thanks again.

On Tue, Sep 2, 2008 at 1:28 PM, Peter Stephenson <pws@xxxxxxx> wrote:
> On Sun, 31 Aug 2008 19:46:14 -0400
> "Rocky Bernstein" <rocky.bernstein@xxxxxxxxx> wrote:
>> I think that the line numbers reported in eval statements are bit hard
>> to explain.
>
> I think they're plain wrong.  The problem is (in the native zsh case where
> the option EVAL_LINENO is set) we reset the line number on an eval, and
> that's all we do.  We really ought to fix up the stack trace so that when
> the line number is reset we generate a new context indicating the surrounding
> line information.
>
> I think this does it.  An "eval" is pretty much like an inline function
> for tracing, with the difference that the line numbering starts from 1
> rather than 0 so needs offsetting when added to the file line number.
>
> I think I've also sorted out a consistency problem with tracing information
> nested inside sourced files.  Note this changes the way $funcsourcetrace
> reports sourced files---they work like autoloaded functions, i.e. the line
> number is 0 because the entire body of the file contains the code.  I ought
> to write a test that checks this.
>
> Index: Doc/Zsh/mod_parameter.yo
> ===================================================================
> RCS file: /cvsroot/zsh/zsh/Doc/Zsh/mod_parameter.yo,v
> retrieving revision 1.11
> diff -u -r1.11 mod_parameter.yo
> --- Doc/Zsh/mod_parameter.yo    13 Aug 2008 21:02:02 -0000      1.11
> +++ Doc/Zsh/mod_parameter.yo    2 Sep 2008 17:15:52 -0000
> @@ -167,7 +167,8 @@
>  vindex(funcfiletrace)
>  item(tt(funcfiletrace))(
>  This array contains the absolute line numbers and corresponding file
> -names for the point where the current function or sourced file was
> +names for the point where the current function, sourced file, or (if
> +tt(EVAL_LINENO) is set) tt(eval) command was
>  called.  The array is of the same length as tt(funcsourcetrace) and
>  tt(functrace), but differs from tt(funcsourcetrace) in that the line and
>  file are the point of call, not the point of definition, and differs
> @@ -176,23 +177,25 @@
>  )
>  vindex(funcsourcetrace)
>  item(tt(funcsourcetrace))(
> -This array contains the file names and line numbers of the
> -points where the functions currently being executed were
> +This array contains the file names and line numbers of the
> +points where the functions, sourced files, and (if tt(EVAL_LINENO) is set)
> +tt(eval) commands currently being executed were
>  defined.  The line number is the line where the `tt(function) var(name)'
>  or `var(name) tt(LPAR()RPAR())' started.  In the case of an autoloaded
> -function in native zsh format where only the body of the function occurs
> -in the file the line number is reported as zero.
> +function  the line number is reported as zero.
>  The format of each element is var(filename)tt(:)var(lineno).
> -For files that have been executed by the tt(source) or tt(.) builtins
> -(in which case there is no separate definition) the trace information is
> -shown as tt(source:0).
> +For functions autoloaded from a file in native zsh format, where only the
> +body of the function occurs in the file, or for files that have been
> +executed by the tt(source) or tt(.) builtins, the trace information is
> +shown as var(filename)tt(:)var(0), since the entire file is the definition.
>
>  Most users will be interested in the information in the
>  tt(funcfiletrace) array instead.
>  )
>  vindex(funcstack)
>  item(tt(funcstack))(
> -This array contains the names of the functions currently being
> +This array contains the names of the functions, sourced files,
> +and (if tt(EVAL_LINENO) is set) tt(eval) commands. currently being
>  executed. The first element is the name of the function using the
>  parameter.
>  )
> Index: Src/builtin.c
> ===================================================================
> RCS file: /cvsroot/zsh/zsh/Src/builtin.c,v
> retrieving revision 1.203
> diff -u -r1.203 builtin.c
> --- Src/builtin.c       31 Aug 2008 13:35:28 -0000      1.203
> +++ Src/builtin.c       2 Sep 2008 17:15:53 -0000
> @@ -4712,15 +4712,53 @@
>  {
>     Eprog prog;
>     char *oscriptname = scriptname;
> -    int oineval = ineval;
> +    int oineval = ineval, fpushed;
> +    struct funcstack fstack;
> +
>     /*
>      * If EVALLINENO is not set, we use the line number of the
>      * environment and must flag this up to exec.c.  Otherwise,
>      * we use a special script name to indicate the special line number.
>      */
>     ineval = !isset(EVALLINENO);
> -    if (!ineval)
> +    if (!ineval) {
>        scriptname = "(eval)";
> +       fstack.prev = funcstack;
> +       fstack.name = scriptname;
> +       fstack.caller = funcstack ? funcstack->name : dupstring(argzero);
> +       fstack.lineno = lineno;
> +       fstack.tp = FS_EVAL;
> +
> +       /*
> +        * To get file line numbers, we need to know if parent is
> +        * the original script/shell or a sourced file, in which
> +        * case we use the line number raw, or a function or eval,
> +        * in which case we need to deduce where that came from.
> +        *
> +        * This replicates the logic for working out the information
> +        * for $funcfiletrace---eval is similar to an inlined function
> +        * call from a tracing perspective.
> +        */
> +       if (!funcstack || funcstack->tp == FS_SOURCE) {
> +           fstack.flineno = fstack.lineno;
> +           fstack.filename = fstack.caller;
> +       } else {
> +           fstack.flineno = funcstack->flineno + lineno;
> +           /*
> +            * Line numbers in eval start from 1, not zero,
> +            * so offset by one to get line in file.
> +            */
> +           if (funcstack->tp == FS_EVAL)
> +               fstack.flineno--;
> +           fstack.filename = funcstack->filename;
> +           if (!fstack.filename)
> +               fstack.filename = "";
> +       }
> +       funcstack = &fstack;
> +
> +       fpushed = 1;
> +    } else
> +       fpushed = 0;
>
>     prog = parse_string(zjoin(argv, ' ', 1));
>     if (prog) {
> @@ -4737,6 +4775,9 @@
>        lastval = 1;
>     }
>
> +    if (fpushed)
> +       funcstack = funcstack->prev;
> +
>     errflag = 0;
>     scriptname = oscriptname;
>     ineval = oineval;
> Index: Src/exec.c
> ===================================================================
> RCS file: /cvsroot/zsh/zsh/Src/exec.c,v
> retrieving revision 1.145
> diff -u -r1.145 exec.c
> --- Src/exec.c  1 Sep 2008 20:18:47 -0000       1.145
> +++ Src/exec.c  2 Sep 2008 17:15:53 -0000
> @@ -4264,10 +4264,16 @@
>     }
>  #endif
>     fstack.name = dupstring(name);
> -    fstack.caller = dupstring(oargv0 ? oargv0 : argzero);
> +    /*
> +     * The caller is whatever is immediately before on the stack,
> +     * unless we're at the top, in which case it's the script
> +     * or interactive shell name.
> +     */
> +    fstack.caller = funcstack ? funcstack->name :
> +       dupstring(oargv0 ? oargv0 : argzero);
>     fstack.lineno = lineno;
>     fstack.prev = funcstack;
> -    fstack.sourced = 0;
> +    fstack.tp = FS_FUNC;
>     funcstack = &fstack;
>
>     if ((shf = (Shfunc) shfunctab->getnode(shfunctab, name))) {
> @@ -4277,8 +4283,7 @@
>        fstack.flineno = 0;
>        fstack.filename = dupstring(fstack.caller);
>     }
> -
> -
> +
>     if (prog->flags & EF_RUN) {
>        Shfunc shf;
>
> Index: Src/init.c
> ===================================================================
> RCS file: /cvsroot/zsh/zsh/Src/init.c,v
> retrieving revision 1.94
> diff -u -r1.94 init.c
> --- Src/init.c  13 Aug 2008 21:02:03 -0000      1.94
> +++ Src/init.c  2 Sep 2008 17:15:53 -0000
> @@ -1061,6 +1061,7 @@
>     unsigned char *ocs;
>     int ocsp;
>     int otrap_return = trap_return, otrap_state = trap_state;
> +    struct funcstack fstack;
>
>     if (!s ||
>        (!(prog = try_source_file((us = unmeta(s)))) &&
> @@ -1100,19 +1101,17 @@
>     trap_state = TRAP_STATE_INACTIVE;
>
>     sourcelevel++;
> -    {
> -       struct funcstack fstack;
> -       fstack.name = dupstring("source");
> -       fstack.caller = dupstring(old_scriptfilename ? old_scriptfilename :
> -                                "zsh");
> -       fstack.flineno = 0;
> -       fstack.lineno = oldlineno;
> -       fstack.filename = fstack.name;
> -       fstack.prev = funcstack;
> -       fstack.sourced = 1;
> -       funcstack = &fstack;
> -    }
> -
> +
> +    fstack.name = scriptfilename;
> +    fstack.caller = funcstack ? funcstack->name :
> +       dupstring(old_scriptfilename ? old_scriptfilename : "zsh");
> +    fstack.flineno = 0;
> +    fstack.lineno = oldlineno;
> +    fstack.filename = scriptfilename;
> +    fstack.prev = funcstack;
> +    fstack.tp = FS_SOURCE;
> +    funcstack = &fstack;
> +
>     if (prog) {
>        pushheap();
>        errflag = 0;
> Index: Src/zsh.h
> ===================================================================
> RCS file: /cvsroot/zsh/zsh/Src/zsh.h,v
> retrieving revision 1.144
> diff -u -r1.144 zsh.h
> --- Src/zsh.h   31 Aug 2008 16:01:12 -0000      1.144
> +++ Src/zsh.h   2 Sep 2008 17:15:53 -0000
> @@ -1077,6 +1077,14 @@
>  #define SFC_COMPLETE 5         /* called from completion code */
>  #define SFC_CWIDGET  6         /* new style completion widget */
>
> +/* tp in funcstack */
> +
> +enum {
> +    FS_SOURCE,
> +    FS_FUNC,
> +    FS_EVAL
> +};
> +
>  /* node in function stack */
>
>  struct funcstack {
> @@ -1086,7 +1094,7 @@
>     char *caller;              /* name of caller */
>     zlong flineno;             /* line number in file */
>     zlong lineno;              /* line offset from beginning of function */
> -    int sourced;               /* type of entry is a sourced file */
> +    int tp;                    /* type of entry: sourced file, func, eval */
>  };
>
>  /* node in list of function call wrappers */
> Index: Src/Modules/parameter.c
> ===================================================================
> RCS file: /cvsroot/zsh/zsh/Src/Modules/parameter.c,v
> retrieving revision 1.47
> diff -u -r1.47 parameter.c
> --- Src/Modules/parameter.c     13 Aug 2008 21:02:04 -0000      1.47
> +++ Src/Modules/parameter.c     2 Sep 2008 17:15:54 -0000
> @@ -583,7 +583,7 @@
>     for (f = funcstack, p = ret; f; f = f->prev, p++) {
>        char *colonpair, *fname;
>
> -       if (!f->prev || f->prev->sourced) {
> +       if (!f->prev || f->prev->tp == FS_SOURCE) {
>            /*
>             * Calling context is a file---either the parent
>             * script or interactive shell, or a sourced
> @@ -595,13 +595,20 @@
>            sprintf(colonpair, "%s:%ld", f->caller, (long)f->lineno);
>        } else {
>            /*
> -            * Calling context is a function; we need to find the line number
> -            * in the file where that function was defined.  For this we need
> -            * the $funcsourcetrace information for the context above,
> +            * Calling context is a function or eval; we need to find
> +            * the line number in the file where that function was
> +            * defined or the eval was called.  For this we need the
> +            * $funcsourcetrace information for the context above,
>             * together with the $functrace line number for the current
>             * context.
>             */
>            long flineno = (long)(f->prev->flineno + f->lineno);
> +           /*
> +            * Line numbers in eval start from 1, not zero,
> +            * so offset by one to get line in file.
> +            */
> +           if (f->prev->tp == FS_EVAL)
> +               flineno--;
>            fname = f->prev->filename ? f->prev->filename : "";
>
>            colonpair = zhalloc(strlen(fname) + (flineno > 9999 ? 24 : 6));
> Index: Test/V06parameter.ztst
> ===================================================================
> RCS file: /cvsroot/zsh/zsh/Test/V06parameter.ztst,v
> retrieving revision 1.3
> diff -u -r1.3 V06parameter.ztst
> --- Test/V06parameter.ztst      12 Aug 2008 20:25:14 -0000      1.3
> +++ Test/V06parameter.ztst      2 Sep 2008 17:15:54 -0000
> @@ -36,4 +36,33 @@
>  >Inside autofn
>  >2 + ./functrace.zsh:21 + ./autofn:0
>  >In sourced file
> ->2 + ./functrace.zsh:22 + source:0
> +>2 + ./functrace.zsh:22 + ./sourcedfile:0
> +
> +  print -r -- 'module_path=(./Modules)
> +  debug_hook() { print $funcfiletrace[1] $functrace[1]; }
> +  set -o DEBUG_BEFORE_CMD
> +  trap "debug_hook" DEBUG
> +  fn() {
> +    a=1
> +    eval "b=2"
> +    c=3
> +  }
> +  fn
> +  w=5
> +  eval "x=6
> +  y=7"
> +  z=8' >rocky3.zsh
> +  $ZTST_testdir/../Src/zsh +Z -f ./rocky3.zsh
> +0:Eval tracing
> +>./rocky3.zsh:5 ./rocky3.zsh:5
> +>./rocky3.zsh:10 ./rocky3.zsh:10
> +>./rocky3.zsh:6 fn:1
> +>./rocky3.zsh:7 fn:2
> +>./rocky3.zsh:7 (eval):1
> +>./rocky3.zsh:8 fn:3
> +>./rocky3.zsh:11 ./rocky3.zsh:11
> +>./rocky3.zsh:12 ./rocky3.zsh:12
> +>./rocky3.zsh:12 (eval):1
> +>./rocky3.zsh:13 (eval):2
> +>./rocky3.zsh:14 ./rocky3.zsh:14
> +
>
> --
> Peter Stephenson <pws@xxxxxxx>                  Software Engineer
> CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
> Cambridge, CB4 0WZ, UK                          Tel: +44 (0)1223 692070
>