Re: Feature request: ZSH_XTRACEFD variable

[Daniel Shahaf <d.s@xxxxxxxxxxxxxxxxxx>]

Timothée Mazzucotelli wrote on Thu, 03 Sep 2020 15:51 +0200:
> Hello, I'm coming back to the ZSH_XTRACEFD feature again :)
> 
> About closing file descriptors or not: I don't understand what it means to
> "leak a FILE".

"FILE", in this context, is the opaque type defined by the C programming
language.  (See the return type of fopen(3), for example.)

In implementations, a FILE typically wraps a "file descriptor" (fd),
which is an int.  The kernel caps the number of file descriptors the
process may have open at any one time (see ulimit/unlimit).  For this
reason it's important to always close fd's when done using them.

To "leak" a fd means to lose track of its integer value and the fact it
needs to be closed.  For example, the following function leaks a fd:
.
    static void
    f(void)
    {
        open("/dev/null", O_WRONLY);
	return;
    }
    
Every call to f() leaks one fd.  If f() were called in a loop,
eventually it would not be possible to open more files:
.
[[[
% repeat $((1 + $(ulimit -n))) { eval "exec {fd$((++i))}>/dev/null" } 
zsh: cannot moved fd 3: too many open files
zsh: exit 1
⋮
]]]

(That's rather dense, I'm afraid, so feel free to ask further.  The
number of iterations is large enough to ensure the error happens in the
last iteration or earlier.  The variable $i is autovivified and used to
generate a different variable name in every iteration of the loop
($fd1, $fd2, …, $fd1025) since I didn't know whether the square
brackets array element variable name syntax could be used in that
context.)

If f() called fopen() rather than open(), it would leak not only a fd
but also whatever other resources the FILE encapsulated (e.g., a small
block of malloc()'d memory).

HTH.  I'll leave the ZSH_XTRACEFD-specific aspects for others.

Cheers,

Daniel

P.S.  The quoted error message is ungrammatical.


> I'm not sure to understand the past comments either: we only use fdopen
> when the value of ZSH_XTRACEFD is > 2.
> For 0, 1 and 2, we re-use the existing file descriptors stdin, stdout and
> stderr.
> 
> Anyway, I added the patch in an attachment. Also, here's the link to the
> commit on my fork:
> https://github.com/pawamoy/zsh/commit/b9b37333fcf02a463f6f742976b37b45ab08742d
> 
> In this patch, I never close any file descriptor.
> 
> There's one last thing that looks weird to me:
> single instructions like ZSH_XTRACEFD=5 are not properly logged in the
> xtrace output.
> It seems they are truncated up to the end of the variable assignment:
> - with a=0 ZSH_XTRACEFD=5, nothing appear in the output either
> - with ZXH_XTRACEFD=5 a=0, only a=0 appears in the output (but no
> +(eval):18 prefix or similar)
> 
> Any idea about this?
> 
> Cheers,
> Timothée
> 
> On Wed, May 6, 2020 at 12:20 AM Bart Schaefer <schaefer@xxxxxxxxxxxxxxxx>
> wrote:
> 
> > (Peter, for some reason Gmail is classifying all email from
> > ntlworld.com as spam, with the notation that it "can't guarantee that
> > this message came from ntlworld.com")
> >
> > On Tue, May 5, 2020 at 9:48 AM Peter Stephenson
> > <p.w.stephenson@xxxxxxxxxxxx> wrote:  
> > >
> > >
> > > The problem is if we fopen() the file descriptor to use stdio as output,  
> > we can either  
> > > leak the entire FILE, not opened by the user, or we can close the entire  
> > FILE.
> >
> > In that case we should be doing the fopen() on a dup() of the
> > descriptor, and fclose()ing the FILE.
> >
> > If it is important that fileno(xtrerr) == $ZSH_XTRACEFD, then we should
> > 1) dup() the descriptor to save a copy
> > 2) fopen() the original
> > 3) after fclose(), dup2() the copy back to the original
> > 4) close() the copy
> >
> > However, I'm not sure it's necessary to be that convoluted.
> >