Support reading from blocking STDIN via child process/thread on Windows

[clue]

Originally, #18 aimed to look into ways to bring native non-blocking I/O to Windows - which still doesn't look like it's going to be supported any time soon unfortunately.

As an alternative, we may use a child process or thread to start blocking read operations on the STDIN stream without blocking the main process.

Here's the gist of this concept:

<?php

use React\Stream\ReadableResourceStream;
use React\EventLoop\Factory;

require __DIR__ . '/../vendor/autoload.php';

$fds = array(
    0 => STDIN,
    1 => array('pipe', 'w'),
    2 => STDERR,
);

$p = proc_open('php ' . escapeshellarg(__DIR__ . '/stdin-child.php'), $fds, $pipes);

$loop = Factory::create();
$stdin = new ReadableResourceStream($pipes[1], $loop);

$stdin->on('data', function ($data) {
    echo "\r";
    var_dump($data);
});

$loop->run();

A simple child process could look something like this:

<?php

while (true) {
    $data = fread(STDIN, 8192);
    if ($data === '' || $data === false) {
        return;
    }

    echo $data;
}

On top of this, we can't really access the STDOUT stream without blocking either, so we may have to use socket I/O instead (see e.g. clue/reactphp-sqlite#13).

On other platforms, we should also avoid inheriting active FDs to the child process (see e.g. clue/reactphp-sqlite#7).

We should be able to use pthreads, libeio or libuv to avoid spawning a child process and use a worker thread instead. This does however require a custom PHP extension to be present.

On top of this, the console will echo each keypress to the output immediately. We may have to disable console echo, but to the best of my knowledge this isn't possible from within PHP either. We may spawn a special binary though, e.g. https://github.com/Seldaek/hidden-input as the C/C++ implementation is relatively straight forward. As an ugly workaround may be able to overwrite the console output using a periodic timer like this:

$overwrite = $loop->addPeriodicTimer(0.005, function () {
    echo "\r" . str_repeat(' ', 40) . "\r";
});
$stdin->on('close', function () use ($overwrite, $loop) {
    $loop->cancelTimer($overwrite);
});

Sounds like fun? 🎉

[spaceemotion]

So I did some investigating on this today, mainly because I tried getting to run https://github.com/spatie/phpunit-watcher on windows (instead of in a VM). While I thought I could get away with no input interaction, it also said the same for STDOUT. Which means, if this problems is to be solved, STDIN+STDOUT both have to be worked on.

[clue]

@spaceemotion You're right, STDOUT could be blocking just like STDIN. The major difference is that on Windows the latter is always blocking whereas the former is only blocking when its buffers are full. This means that for most applications that only print less than a few kilobytes of output, the STDOUT can be considered non-blocking for normal console applications.

In a gist, this means you can use this piece of code even on Windows to create an output stream that is mostly non-blocking like this:

$stream = new ThroughStream(function ($data) {
     echo $data;
});

Under the hood, echo will invoke the equivalent of fwrite(STDOUT, $str), so this may still block once you're printing a lot of data.

[Nek-]

I tried this crappy code:

<?php

$forwardInputScript = '<?php
file_put_contents("yolo.txt", "");
while (true) {
    $data = fread(STDIN, 1);
    if ($data === \'\' || $data === false) {
        return;
    }
    if ($foo = trim($data)) {
        $content = "";
        if (file_exists("yolo.txt")) {
            $content = file_get_contents("yolo.txt");
        }
        file_put_contents("yolo.txt", $content . $data);
        echo $data;
    }
}';

$filename = sys_get_temp_dir() . DIRECTORY_SEPARATOR . 'forward_input.php';
file_put_contents($filename, $forwardInputScript);

$fds = [
    0 => STDIN,
    1 => ['socket'],
    2 => STDERR,
];

$p = proc_open('php ' . escapeshellarg($filename), $fds, $pipes);


foreach ($pipes as $i => $pipe) {
    var_dump($i);
    var_dump(stream_set_blocking($pipe, false));
}


sleep(3);
$r = $pipes;
$w = null;
$e = null;

if (@stream_select($r, $w, $e, 0, 0)) {
    $something = stream_get_contents($r[1]);
} else {
    $something = '';
}

echo "First echo \n";
var_dump($something);
sleep(3);

$r = $pipes;
$w = null;
$e = null;
if (@stream_select($r, $w, $e, 0, 0)) {
    $something = stream_get_contents($r[1]);
} else {
    $something = '';
}

echo "Second echo \n";
var_dump($something);
sleep(3);

echo "fin\n";

It doesn't work. Actually what happens is super weird. It sometimes work partially: the input is forward partially (or entirely sometimes) to the subprocess, I know it because it prints into a file (called yolo.txt). Besides, I have nothing from the socket until I press enter... Ok, it's not blocking anymore but if enter is required to make it work, it's useless.

I suggest you to test it on your side so you get what I'm trying to explain here :') .

[clue]

@Nek- This looks about right and seems to confirm everything that's been discussed in here so far 👍 Your code makes sure the main parent process is not blocked when the child process reads from STDIN.

However, the child process will still use a blocking read. On top of this, it will only complete when a full line has been buffered (on enter). I'm not aware of any workarounds for this on Windows using PHP.

There are a number of ways to read from console input on Windows without blocking, but this requires lower level interfaces not available in PHP. For instance, take a look at https://github.com/Seldaek/hidden-input, the code is actually pretty straight forward.

This combined means that you should in fact be able to launch a (non-php) binary that reads individual characters from the console input and then sends it over a socket the main PHP process. I'm not aware of any way to make this work with pure PHP.

If you're feeling adventurous, you may want to take a look at FFI on Windows to directly access the underlying system APIs. 👍

[Nek-]

Hey guys, just so you know, I managed to make it (non-blocking stdin on Windows) work with FFI and the Windows API. You can see the definition and a PHP example in this gist :
https://gist.github.com/Nek-/118cc36d0d075febf614c53a48470490

[clue]

@Nek- Thanks for reporting back with this the results, this is some lovely piece of code! 😃

Have you looked into how this could be combined with an event loop? Right now it appears to be using the GetNumberOfConsoleInputEvents() API which needs to be called "regularly"?

<?php

define('STD_INPUT_HANDLE', -10);

// https://docs.microsoft.com/fr-fr/windows/console/setconsolemode
define('ENABLE_ECHO_INPUT', 0x0004);
define('ENABLE_PROCESSED_INPUT', 0x0001);
define('ENABLE_WINDOW_INPUT', 0x0008);

// https://docs.microsoft.com/fr-fr/windows/console/input-record-str
define('KEY_EVENT', 0x0001);

$windows = \FFI::load('windows.h');


$handle = $windows->GetStdHandle(STD_INPUT_HANDLE);

$oldMode = $windows->new('DWORD');
if(!$windows->GetConsoleMode($handle, \FFI::addr($oldMode))) {
    echo "Failure A\n";
    exit;
}

$newConsoleMode = ENABLE_WINDOW_INPUT | ENABLE_PROCESSED_INPUT;
if (!$windows->SetConsoleMode($handle, $newConsoleMode)) {
    echo "Impossible to change the console mode\n";
    exit;
}


function printToCoordinates(int $x, int $y, string $text) {
    //fprintf(STDOUT,"\x1b7\x1b[".$y.';'.$x.'f'.$text."\x1b8");
    fprintf(STDOUT, "\033[%d;%dH%s", $y, $x, $text);
}

$i = 0;
$bufferSize = $windows->new('DWORD');
$s = '*';

$arrayBufferSize = 128;
$inputBuffer = $windows->new("INPUT_RECORD[$arrayBufferSize]");
$cNumRead = $windows->new('DWORD');
fprintf(STDOUT,"\033[H\033[J");

while ($i < 60) {
    printToCoordinates($i, 5, $s);
    $i++;


    $windows->GetNumberOfConsoleInputEvents(
        $handle,
        \FFI::addr($bufferSize)
    );

    if ($bufferSize->cdata > 1) {
        if (! $windows->ReadConsoleInputW(
                $handle,                   // input buffer handle
                $inputBuffer,                  // buffer to read into
                $arrayBufferSize,          // size of read buffer
                \FFI::addr($cNumRead)) ) { // number of records read
            echo "Read console input failing\n";
            exit;
        }

        for($j = $cNumRead->cdata - 1; $j >= 0; $j--) {
            if ($inputBuffer[$j]->EventType === KEY_EVENT) {
                $keyEvent = $inputBuffer[$j]->Event->KeyEvent;
                if ($keyEvent->uChar->AsciiChar === 'a') {
                    echo "You pressed A\n";
                    exit;
                }
            }
        }
    }

    usleep(100000);
}


$windows->CloseHandle($handle);

#define FFI_LIB "C:\\Windows\\System32\\kernel32.dll"
// Does FFI work on windows ? https://github.com/dstogov/php-ffi/issues/15


// This is a microsoft specific type, here is its definition for gcc
// https://github.com/Alexpux/mingw-w64/blob/d0d7f784833bbb0b2d279310ddc6afb52fe47a46/mingw-w64-headers/crt/time.h#L36
typedef unsigned short wchar_t;

// Source for data correpsondance
// https://docs.microsoft.com/en-us/windows/win32/winprog/windows-data-types

typedef int BOOL;
typedef unsigned long DWORD;
typedef void *PVOID;
typedef PVOID HANDLE;
typedef DWORD *LPDWORD;
typedef unsigned short WORD;
typedef wchar_t WCHAR;
typedef short SHORT;
typedef unsigned int UINT;
typedef char CHAR;

typedef struct _COORD {
  SHORT X;
  SHORT Y;
} COORD, *PCOORD;

typedef struct _WINDOW_BUFFER_SIZE_RECORD {
  COORD dwSize;
} WINDOW_BUFFER_SIZE_RECORD;

typedef struct _MENU_EVENT_RECORD {
  UINT dwCommandId;
} MENU_EVENT_RECORD, *PMENU_EVENT_RECORD;

typedef struct _KEY_EVENT_RECORD {
  BOOL  bKeyDown;
  WORD  wRepeatCount;
  WORD  wVirtualKeyCode;
  WORD  wVirtualScanCode;
  union {
    WCHAR UnicodeChar;
    CHAR  AsciiChar;
  } uChar;
  DWORD dwControlKeyState;
} KEY_EVENT_RECORD;

typedef struct _MOUSE_EVENT_RECORD {
  COORD dwMousePosition;
  DWORD dwButtonState;
  DWORD dwControlKeyState;
  DWORD dwEventFlags;
} MOUSE_EVENT_RECORD;

typedef struct _FOCUS_EVENT_RECORD {
  BOOL bSetFocus;
} FOCUS_EVENT_RECORD;

typedef struct _INPUT_RECORD {
  WORD  EventType;
  union {
    KEY_EVENT_RECORD          KeyEvent;
    MOUSE_EVENT_RECORD        MouseEvent;
    WINDOW_BUFFER_SIZE_RECORD WindowBufferSizeEvent;
    MENU_EVENT_RECORD         MenuEvent;
    FOCUS_EVENT_RECORD        FocusEvent;
  } Event;
} INPUT_RECORD;
typedef INPUT_RECORD *PINPUT_RECORD;

// Original definition is
// WINBASEAPI HANDLE WINAPI GetStdHandle (DWORD nStdHandle);
// https://github.com/Alexpux/mingw-w64/blob/master/mingw-w64-headers/include/processenv.h#L31
HANDLE GetStdHandle(DWORD nStdHandle);

// https://docs.microsoft.com/fr-fr/windows/console/getconsolemode
BOOL GetConsoleMode(
	/* _In_ */HANDLE  hConsoleHandle,
	/* _Out_ */ LPDWORD lpMode
);

// https://docs.microsoft.com/fr-fr/windows/console/setconsolemode
BOOL SetConsoleMode(
  /* _In_ */ HANDLE hConsoleHandle,
  /* _In_ */ DWORD  dwMode
);

// https://docs.microsoft.com/fr-fr/windows/console/getnumberofconsoleinputevents
BOOL GetNumberOfConsoleInputEvents(
  /* _In_ */  HANDLE  hConsoleInput,
  /* _Out_ */ LPDWORD lpcNumberOfEvents
);

// https://docs.microsoft.com/fr-fr/windows/console/readconsoleinput
BOOL ReadConsoleInputA(
  /* _In_ */  HANDLE        hConsoleInput,
  /* _Out_ */ PINPUT_RECORD lpBuffer,
  /* _In_ */  DWORD         nLength,
  /* _Out_ */ LPDWORD       lpNumberOfEventsRead
);
BOOL ReadConsoleInputW(
  /* _In_ */  HANDLE        hConsoleInput,
  /* _Out_ */ PINPUT_RECORD lpBuffer,
  /* _In_ */  DWORD         nLength,
  /* _Out_ */ LPDWORD       lpNumberOfEventsRead
);

BOOL CloseHandle(HANDLE hObject);

[Nek-]

I didn't try to combine it to an event loop, but it probably needs to have its own event loop.

GetNumberOfConsoleInputEvents() was surprisingly not required in my tests with PHP because ReadConsoleInputW() always returned "\0" (which is fine).

But all of this comes from another test in C (containing some comments) and when I tested it outside of PHP, the call to GetNumberOfConsoleInputEvents() was clearly required: ReadConsoleInputW() is blocking when the input buffer is empty.