Sunday, March 17, 2013

JAVASCRIPT setTimeout(fn, 0) for batching requests to a server

I was working on a web application where a user would subscribe for ticket prices for air-fares. She would enter the source and the destination and get live updates to prices and options.
Now, every time she enters a new pair, a new panel/tile would be added to her page that started ticking. Those panels were persistent; she could logout and when she logged back in later, the panels would load and start the subscriptions anew.
The browsers we needed to support included IE8. We were using socket.io for making the requests and receiving subscriptions. But the library would fall back to using jsonp-long polling or xhr-streaming for instance. Now, as we know, IE limits the number of Http connections one can have to 6. This meant if we made too many requests concurrently, we would experience delays as the request would get queued up. This was the case when loading a page with numerous persisted panels.
We thus allowed a batched request API, a simple array, e.g.
[ "NewYork-London, "NewYork-Paris", "NewYork-Jakarta" ]
It would be nicer to not have to differentiate between the user entering 3 individual pairs or a batched request being made on her behalf upon page load.
This is where setTimeout(..., 0) becomes useful. [Note: Underscore.js's _.defer() does the same]
You can look at the code in action at CLICK HERE
function Service (socket) {
  this.socket = socket;
  this.liveStreams = {};
  this.pendingSubscribe = [];
  socket.on('response', function (data) {
    // Assume response is a json object,e.g.
    //  {
    //    pair:'NewYork-London',
    //    data: {
    //      cost: '560',
    //      class: 'Economy',
    //      departing: 'EWR',
    //      arrving: 'LHR'
    //    }
    //  }
    data = JSON.parse(data);
    this.liveStreams[data.pair].call(null, data.update);
  }.bind(this));
}
   
function flush (service) {
  console.log('FLUSHING ' + service.pendingSubscribe.join(','));
  service.socket.emit('request', service.pendingSubscribe);
  // reset for next flushSubscribe
  service.pendingSubscribe = [];
}

Service.prototype.subscribe = function (pair, callback) {
  // in case of intial page load, this "deferring" will force batching
  if(this.pendingSubscribe.length === 0) setTimeout(flush, 0, this);

  this.pendingSubscribe.push(pair);
  this.liveStreams[pair] = callback;
};
When a number of requests are made in a loop (that is in the same call stack), the FIRST subscribe() call schedules a call to flushSubscribe() to be made when the current stack unrolls (not exactly! See the bottom of this post). All subsequent subscribe() calls in the loop will NOT unroll the stack and thus simply modify the payload, namely pendingSubscribe.

Now, if you are wondering why we don't just prepare the batched request where we loop, it is because when you are using a framework like Backbone, you want to have a layered architecture and that might not be easy. In my case, the panel was designed to be editable and would issue its own request; consequently, it would be unaware of any batching strategy.

Note: setTimeout(fn, 0) simply puts the function fn in the event queue's front but after any other similar functions already scheduled with setTimeout(otherFn, 0). CLICK HERE FOR PROOF. This doesn't affect the batching logic above per se. However, if we had an unsubscribe() call associated with a button click - then we'd need to be a bit craftier; as we don't want to send an unsubscribe request BEFORE the corresponding subscribe request. If anyone needs help with that, drop me a comment and I can update this post with a more complete example. But in brief, the unsubscribe request would check if we already have a pendingSubscribe, if so simple do an "early cancellation", i.e. remove the pair from that array.

1 comment:

  1. We were at http://www.meetup.com/nyc-node-js/events/109739072/ and it was mentioned how 0.10.0 node.js's process.nextTick() was now going to be executed as soon as the current stack unrolls. I find process.nextTick() comparable to setTimeout(fn, 0) but this new behavior leads to a nuance.

    ReplyDelete