Working with TCP Protocol in Node.JS

Node has an out-of-the-box TCP server implementation. This can be accessed through the 'net' module.

Creating a TCP server:

var net = require('net');
var server = net.createServer(function(socket) {
//code to handle each connection
});

This is fairly similar to HTTP server in Node. The createServer creates a Server object and takes in a function callback that will be called to handle each connection. This function will be supplied the Socket object of the incoming connection. It is a two-way stream object, meaning you can send as well as receive data with it.

net.Server is an EventEmitter, which means that it emits certain events. They are:
listening - when it starts listening for connections
connection - when a connection is received. Handling this event is same as passing a function callback to createServer()
close - when a server is closed and not bound to any port
error - when an error occurs at a server level

Socket is a Stream
As the Socket object is a bi-directional Stream, it can be read from or written to. It emits data and end events whenever receiving data. And you can write data to it using write() function. You can also tell the server to close the connection by calling end().
You can also performing piping operations involving the Socket stream. The following code, for example, sends the contents of a text file to the Socket by piping:

var net = require('net');
var fs = require('fs');

var server = net.createServer(function(socket){
var fileStream = fs.createReadStream('test.txt');
fileStream.pipe(socket, {end: false});
});

server.on('error', function(e){
console.log("Server error: " + e.message);
});

server.listen(3001);

Streams in Node.JS

Streams are an abstraction that allow us to have a simple interface with an object that either provides or consumes data. There are 2 types of streams:
1) Read: These are the streams that connect a data provider to our app. We can read from them.
2) Write: These streams connect a data consumer to our app. We can write to them.

There are various ways in which streams can be created. Sometimes they are just handed over to the application. All the outside interfaces (file I/O, networking) are available in the form of streams.

Streams have various events:
1) data: These events signify that a read stream has sent a data packet. We can listen to this event to consume the data sent by the provider.
2) end: This event signifies that the incoming data has ended and now no data will be sent.
3) drain: This event is only for writable streams. When Node writes to streams, it may not write it immediately but store in a buffer. When the buffer is finally drained and all the data is written, this event is raised.

Stream functions:
1) write(buffer | string[, encoding]): This function writes the contents of the buffer or the string to the stream. An optional parameter can be specified with strings to indicate their encoding. The default is UTF-8. The return value of this function is interesting. If it returns true, then the write event was flushed from the buffer. If false, then the data was queued in the buffer.

2) pause() :This function will pause a readable stream. This means that the stream will no longer raise 'data' events. The pause function is implemented differently by different types of streams.

3) resume(): This is the opposite of pause() function. It starts the flow of data from the stream again.

Case study: The slow client problem
Node has non blocking I/O, which means that if a writable stream cannot be flushed instantly, it will insert the data into it's queue and continue with the event loop. This happens particularly when the read stream is faster than the write stream. If you have a http server which reads from a file stream and writes to the HTTP response stream, then this problem can occur. It will add to the memory usage of the program. The more the number of requests, the severe will be the problem. This can be solved by the above functions.

http.createServer(function(req, res) {
var stream = fs.createReadStream('sample.txt');
stream.pipe(res);
stream.on('data', function(data) {

console.log('data: ' + data);
if(!res.write(data)) {
stream.pause();
}
});

stream.on('drain', function() {
stream.resume();
})

stream.on('end', function() {
res.end();
});

}).listen(3000);

In the above example, we pause the read stream whenever res.write() returns false, meaning that it could not drain immediately. Then, when it's drained, we resume the stream again.

The pipe() function
The above solution to the slow client problem is a very commonly occuring pattern in Node, and there is a single function that can take care of this. It's called the pipe() function. The above tasks can be done using pipe() function as follows:

stream.pipe(res);

The pipe function is called on the source stream. It takes a parameter as the stream to write to. And it automatically passes the data between streams, while taking care of the slow client problem by pausing and resuming whenever necessary.