commit 52ecc76c7fa1865603f27bc838efaeaa03cad77c
author GuyBrush <miguel.barro@live.com> Wed Feb 21 20:16:38 2024 +0100
committer Christian Brabandt <cb@256bit.org> Wed Feb 21 20:16:38 2024 +0100
tree 6bf0b671bd102a7537dc1320bcfa52d26e3d7ba9
parent 9ca335aad295fdaf4b75d3969b093422c1cc48ee

patch 9.1.0123: MS-Windows: system() may deadlock

Problem:  MS-Windows: system() may deadlock when calling binaries that
          expect stdin
Solution: Ignore the SHELL_EXPAND flag
          (GuyBrush)

This happens on binaries that expect stdin. For example:

:echo system("xxd")

will cause a deadlock.

SHELL_EXPAND is a flag devoted to support the linux implementation of
the backtick-expansion mechanism.

On linux backtic-expansion relies in the function mch_expand_wildchars()
(os_unix.c) that delegates on each specific shell (bash, sh, csh, zsh)
the expansion. Basically it composes a shell command that does the
expansion and redirects the output to a file and call_shell() it. On
windows backtick-expansion is performed by Vim itself.

On linux SHELL_EXPAND modifies how mch_call_shell_fork() (os_unix.c)
works. This function:

- relies on posix fork() to spawn a child process to execute a
  external command.
- Child and parent process communicate using pipes (or pseudoterminal
  if available).
  User input (type ahead content) is processed in a loop only if
  !(SHELL_EXPAND || SHELL_COOKED).
  Though signals are used to detect Ctrl-C in all cases (the input
  loop is not necessary to interrupt the function).
  In the backtick-expansion the external command is the shell command
  that provides the expansion. For the child redirection:
  -  SHELL_EXPAND replaces stdin, stdout & stderr to /dev/null. This is
     why the shell command composed includes redirection (otherwise
     output would be lost).

  -  !SHELL_EXPAND replaces stdin, stdout & stderr with the parent
     created pipes (or pseudoterminal).
     Note that the use of SIGINT signal prevents mch_call_shell_fork()
     from hanging vim.

On Windows mch_system_piped() (os_win32.c) (which is only used when the
GUI is running) mimics mch_call_shell_fork() (os_unix.c).
Win32 lacks fork() and relies on CreateProcessW() and only has pipe
support (not pseudoterminal) which makes the implementation much
different.

But, the key idea is that windows lacks signals, the OS provides support
for console apps but gvim is not one. The only way of detecting a Ctrl-C
is actually processing user input (type ahead content). By ignoring the
user input under SHELL_EXPAND the function can hang gvim.

Ignoring SHELL_EXPAND flag has no consequence in Windows because as
mentioned above it is only meaningful in linux.

closes: #13988

Signed-off-by: GuyBrush <miguel.barro@live.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>