Promise chain with subtle asynchronous misordering

357
June 30, 2017, at 12:37 PM

A promise chain is useful for not over-writing a stream in node (see Bergi's comment). But here's an example that in subtle asynch circumstances produces mis-ordered writes on the write stream:

var fs = require( 'fs' );
function WTest() {
  this.writeStream = fs.createWriteStream( 'junkTest.txt' );
  this.readyToWrite = Promise.resolve( true );
};
WTest.prototype.writeNext = function writeNext( text, callNum ) {
  this.readyToWrite = this.readyToWrite
    .then( status => writeToStream( text, this.writeStream, status, callNum ) );
}
var wTest = new WTest();
for ( var i = 0; i < 10; i++ ) wTest.writeNext( 
  `${i}, some line of text that might be xxxxxxxxxxxxxxxxxxxxxxxx long \n`, i );
function writeToStream( text, writableStream, readyToWrite, callNum ) {
  return new Promise( ( resolve ) => {
    if ( readyToWrite ) resolve( writableStream.write( text ) );
    else writableStream.once( 'drain', () => resolve( writableStream.write( text ) ) );
  } );
}

If you run it as is in node 7.8, it works. But in complicated asynch circumstances, it fails, e.g. one line is written out of order. What's asynchronously dangerous about the above code? Can you create an asynch example where it fails?

My instance of failure includes multiple async file reads with the above code serving a logger function. I've verified that the calls to writeNext are arriving in the expected order by writing them to the console immediately after the call to the writeNext function. As mentioned in a comment, I've written with both (both direct to the same file), with the direct "just write" write (-) immediately after the promises write (=). The line sequence in the output file is 1-,1=,2-,3-,4-,5-,6-,7-,8-,2=,7=,3=,4=,5=,6=,9-,9=,10-,10=

Big picture wisdom from Bergi: "The only way to prevent over-writing is to stop generating new chunks." So the above is only of technical interest. Creating a promise chain worsens the memory cost of buffering.

Answer 1

The line sequence in the output file is
1-,1=,2-,3-,4-,5-,6-,7-,8-,2=,7=,3=,4=,5=,6=,9-,9=,10-,10=

The code from your question won't do that when there's only a single WTest instance and nothing else writes into that file.

What's asynchronously dangerous about the above code? Can you create an asynch example where it fails?

The only problem I see is that you might miss a drain event since you only start listening when there's another chunk to write instead of immediately after you got false, but that would lead to a forever-hanging stream and not to lines written out of order.

Better:

function WTest(filename) {
  this.writeStream = fs.createWriteStream(filename);
  this.readyToWrite = Promise.resolve();
}
WTest.prototype.writeNext = function writeNext(text, callNum) {
  this.readyToWrite = this.readyToWrite.then(() => {
    const status = writeStream.write(text);
    if (!status)
      return new Promise(resolve => this.writeStream.once('drain', resolve));
  });
};
Rent Charter Buses Company
READ ALSO
Catching errors from spawned Node.js process

Catching errors from spawned Node.js process

Here is an example where child process error is not fired:

311
Is there a way to compile a groovy script in Javascript during runtime? [on hold]

Is there a way to compile a groovy script in Javascript during runtime? [on hold]

I'm developing a website in Javascript using Nodejs, but one of the requirements is that users can upload Groovy scripts

266
Python - passing csrf token with requests to a node.js server

Python - passing csrf token with requests to a node.js server

I want trying to login to nodejs server using csrf token, but it is not working and I am confused which csrf token to consider

457
PHP or Javascript, How to take any text data on notepad file automatically

PHP or Javascript, How to take any text data on notepad file automatically

I have a Cisco log txt / notepad file which used to create a Preventive Maintenance Report

255