Working with TCP Protocol in Node.JS

Node has an out-of-the-box TCP server implementation. This can be accessed through the 'net' module.

Creating a TCP server:

var net = require('net');
var server = net.createServer(function(socket) {
//code to handle each connection
});

This is fairly similar to HTTP server in Node. The createServer creates a Server object and takes in a function callback that will be called to handle each connection. This function will be supplied the Socket object of the incoming connection. It is a two-way stream object, meaning you can send as well as receive data with it.

net.Server is an EventEmitter, which means that it emits certain events. They are:
listening - when it starts listening for connections
connection - when a connection is received. Handling this event is same as passing a function callback to createServer()
close - when a server is closed and not bound to any port
error - when an error occurs at a server level

Socket is a Stream
As the Socket object is a bi-directional Stream, it can be read from or written to. It emits data and end events whenever receiving data. And you can write data to it using write() function. You can also tell the server to close the connection by calling end().
You can also performing piping operations involving the Socket stream. The following code, for example, sends the contents of a text file to the Socket by piping:

var net = require('net');
var fs = require('fs');

var server = net.createServer(function(socket){
var fileStream = fs.createReadStream('test.txt');
fileStream.pipe(socket, {end: false});
});

server.on('error', function(e){
console.log("Server error: " + e.message);
});

server.listen(3001);

Streams in Node.JS

Streams are an abstraction that allow us to have a simple interface with an object that either provides or consumes data. There are 2 types of streams:
1) Read: These are the streams that connect a data provider to our app. We can read from them.
2) Write: These streams connect a data consumer to our app. We can write to them.

There are various ways in which streams can be created. Sometimes they are just handed over to the application. All the outside interfaces (file I/O, networking) are available in the form of streams.

Streams have various events:
1) data: These events signify that a read stream has sent a data packet. We can listen to this event to consume the data sent by the provider.
2) end: This event signifies that the incoming data has ended and now no data will be sent.
3) drain: This event is only for writable streams. When Node writes to streams, it may not write it immediately but store in a buffer. When the buffer is finally drained and all the data is written, this event is raised.

Stream functions:
1) write(buffer | string[, encoding]): This function writes the contents of the buffer or the string to the stream. An optional parameter can be specified with strings to indicate their encoding. The default is UTF-8. The return value of this function is interesting. If it returns true, then the write event was flushed from the buffer. If false, then the data was queued in the buffer.

2) pause() :This function will pause a readable stream. This means that the stream will no longer raise 'data' events. The pause function is implemented differently by different types of streams.

3) resume(): This is the opposite of pause() function. It starts the flow of data from the stream again.

Case study: The slow client problem
Node has non blocking I/O, which means that if a writable stream cannot be flushed instantly, it will insert the data into it's queue and continue with the event loop. This happens particularly when the read stream is faster than the write stream. If you have a http server which reads from a file stream and writes to the HTTP response stream, then this problem can occur. It will add to the memory usage of the program. The more the number of requests, the severe will be the problem. This can be solved by the above functions.

http.createServer(function(req, res) {
var stream = fs.createReadStream('sample.txt');
stream.pipe(res);
stream.on('data', function(data) {

console.log('data: ' + data);
if(!res.write(data)) {
stream.pause();
}
});

stream.on('drain', function() {
stream.resume();
})

stream.on('end', function() {
res.end();
});

}).listen(3000);

In the above example, we pause the read stream whenever res.write() returns false, meaning that it could not drain immediately. Then, when it's drained, we resume the stream again.

The pipe() function
The above solution to the slow client problem is a very commonly occuring pattern in Node, and there is a single function that can take care of this. It's called the pipe() function. The above tasks can be done using pipe() function as follows:

stream.pipe(res);

The pipe function is called on the source stream. It takes a parameter as the stream to write to. And it automatically passes the data between streams, while taking care of the slow client problem by pausing and resuming whenever necessary.

The Journey Begins

So after days of pondering and finding what to learn next, I zeroed upon JavaScript and the MEAN stack. For thos uninitiated, let me tell what MEAN stack is: It is a set of technologies that can be used to build web applications. Each part (MongoDB, ExpressJS, AngularJS, NodeJS) plays a specific role and has a specific place. Also as you may have guessed from the names, all are written entirely in JavaScript.

 

Now I have used JS previously in all my projects, extensively in CodeDiary and a PhoneGap app that built for someone I was working for, I still felt that I am not fully acquainted with even the basics of JavaScript and hence needed to learn more deeper. 

Also for a brief period I tried FreeCodeCamp (It's awesome) however I could not pursue it further as I felt the syllabus was too huge and was targeted towards total beginners in the game. Though I was fully convinced by Quincy Larsson's JS evangelism and decided that the MEAN stack will be my next gig.

JavaScript is no longer confined in the browser. It has moved ahead of all the criticism and hate thrown at it. Today, JavaScript is used in following ways:

• Conventionally, in the browser, to add behaviour to pages
• Outside the browser, as a stand alone engine to execute scripts on server side (NodeJS)
• In IoT devices such as Raspberry Pi for interacting with hardware
• The concepts stemmed from JS such as JSON are used in NoSQL databases 

So, it is very much possible to make a complete product entirely with JS right from the database tier to UI. This is what appealed to me (besides the colorful logos and slick websites of each component in the MEAN stack).

Last, a little mention about this very blog that you're reading. I read somewhere (most probably Quora) that maintaining a blog about what you are learning and keeping it updated is the best way to keep yourself motivated and document your learning at the same time. I'm a big fan of recording whatever I have learnt so that I can go back to it if the need be. Hence this blog.

I hope, someday (which I hope, is not too far), I will be reasonably proficient in the entire MEAN stack and develop slick products using it.

JavaScript 101: Variable Lifetime and Scoping

So as I said in my previous post, I decided to get re-introduced to JavaScript in a very in-depth way. Searched for a book on the internet. Found that Head First JavaScript is a good one which covers all the basic and some advanced topics and does so in a fun and engaging way. Decided to go with it.

Variable Scoping and Lifetime

This is probably the most read topics of all time whenever learning any language. Still, it is also the most ignored topic of all time. I did read the rules for scoping and lifetime for all the languages I learnt C, Java, but somehow the entire thing didn’t stick. I decided to once and for all “get” this and move on.

There are basically two types of scopes:

1. Global: declared outside functions
2. Local: declared in functions

That’s it. It’s that simple. Also, we should always use keyword var while declaring variables, else declarations (even those in functions) will be considered global! Major pitfall.

JavaScript does not have block level scope

Yes! Consider the following code in C/Java:

for (int i = 0 ; i < x ; i++) {}

Here, i is a variable which is declared locally to the loop. After the loop exits, there is no way you can retrieve the value of i.

However, consider the functionally same code in JS:

for (var i = 0 ; i < x ; i++) {}

In this code, the declared variable i is a global variable! Mind = blown!
This is because, as stated earlier, JS has only 2 types of scope: function (local) and global. So to solve the above mystery, you need to ask yourself: Is i in a function? No? Then it’s global.

Shadowing

If you declare a variable with the same name as a global variable in a function, the local variable always gets priority over the global one. The global one is thus inaccessible while you are inside the function. Same goes for parameter names, which are also local variables. Better avoid same names for local and global vars.

Hoisting

JavaScript fancy word #1. :-D Basically, hoisting means that all variables, no matter wherever they are declared or initialized, are always ‘hoisted’ (lifted) to the top of the script, except if they are in functions. Thus, you can do this:

foo = “bar”;
var foo;

That means you can use a variable before it is even declared. Note: Only declarations are hoisted. Initializations/assignments are not. They execute on the line they were written originally.

It’s not the best practice to make extensive use of hoisting as it will eventually lead to confusing code.

JavaScript 101: Object Oriented JS

Reading the chapter related to OOP in Head First JS. Some basic rules:

Objects can be created as follows:

var myCar = {

 color: “Red”,
 miles: 1321,
 model: “Honda”
};

It’s that simple.

Object properties can be accessed as follows:

myCar.color 

will yield “Red”.

No object can contain two or more properties with same names.

If you set a value of a property that doesn’t (yet) exist, it will be created. Like:

myCar.owner: “Me”; 

will not throw an error, instead it will add the property owner to myCar variable.

Object properties can be deleted as follows:

delete myCar.miles; 

will simply drop the miles property from myCar object. delete returns a boolean value, which is true if it deleted the property or the property doesn’t exist, false if it encountered an error in deleting the property.

Objects don’t hold actual data, they hold references:

This is similar to Java. There are only a few primitive types: number, string, and boolean. A variable can contain this type of data directly. However, for objects and arrays, we have what is called as a reference type: it holds an address to the actual object. This is because we know beforehand the size of number, string and boolean variables. However the objects can contain 100’s of properties, so there’s no way to know their size beforehand. The object variables contain a reference to the actual object. However JS Engine abstracts the process of accessing the referenced object and makes it transparent to the user. Thus, normal and object variables can be accessed in a similar way.

Object are passed around as references:

Passing an object to a function isn’t same as passing a normal variable. JS is pass by value, or pass by copy, which means the variable will be copied and passed to the function. With objects, as we know that object variables hold references and not actual objects. Therefore, while passing to a function, the copy of the reference is passed, however the reference still points to the same object. Thus, a change made to the object in a function will change the original object.

var tito = {

  name: "Tito",
  age: 3
};
 
function growDog(dog) {
  dog.age++;
}
 
document.write(dog.name + " is " + tito.age + " years old.");
 
growDog(tito);
 
document.write(dog.name + " is " + tito.age + " years old.");

Will output: Tito is 3 years old.Tito is 4 years old.

 

This pointer is mandatory:

To access object properties from within the methods of the object, you must access it using a this pointer.

this is set to point to the current object when the method is called.

Ways of accessing a property:

Properties can be accessed in two ways:

object.propname 

Or

object[propname] 

These 2 forms are equivalent. The second form can accept any expression as long as it evaluates to a property name. Example:

object [“prop” + “name”] 

This cannot be done with the first method.

JavaScript 101: Data Types

The Head First JavaScript book says ..

…but to truly master the language, get that promotion, and get on to the things you really want to do in life, you have to rock at types.

I guess that’s what I really want to do .. So better study the chapter on Types seriously.

Null, Undefined, NaN:

This is a very confusing topic, even after so many years of programming I can’t wrap it up around my head! So in a nutshell -

Null is used to represent “no object”. That is, it is used in place where an object should be there but it isn’t. Like getElementById returns null when it does not find an object with that ID.

Undefined, on the other hand, is used to represent a variable that has not been initialized, an object with a missing property, or an array with a missing value.

NaN is a special value in JS that is used to represent a number that can’t be represented in computers. One example is (0/0). NaN != NaN, because the two numbers which can’t be represented may no necessarily be equal. IsNaN() is a special function used to check whether or not an expression evaluates to NaN.

Type of NaN is : number

Type of undefined is : undefined

Type of null is: object

Equality Checking and Type Conversion:

JavaScript has the == operator to check for equality. This operator works as follows:

1. If the types of two operands are same, compare them.
2. If the types of two operands are different, try to convert them to same types and then compare them.

It uses the following rules to convert operands:

1. When comparing a string and number, convert the string to a number. If it cannot be converted, it is a NaN.
2. When converting any value to boolean, convert both of them to numbers. (False = 0, True = 1) and compare.
3. Comparing null and undefined always yields true.

Type Conversion for + operator:

If both operands are numbers, add. Else, if one is a string, concat them.

Type Conversion for other operators:

Try to convert both operands to numbers and then evaluate -, * or /.

The strict equality operator:

JavaScript uses the operator === to ensure strict comparison. This operator works just like == operator of any strictly typed language: if both operands ain’t of the same type, they ain’t equal.

Checking whether objects are equal:

Both == and === behave exactly same when comparing object references. They both compare references, which means the variables containing the addresses. So obj1 == obj2 returns true, if and only if obj1 and obj2 point to the same object.

Truthy and Falsy values:

These values are those which are not boolean but which are considered to be “true” or “false” by JavaScript, when comparing. There are 5 falsy values, rest all are truthy:

1. null
2. undefined
3. “”
4. 0
5. NaN

A closer look at strings:

JavaScript has two types of strings: primitive and object strings. When we create a string, by default it is primitive. Such a string does not have any functions or properties. However, whenever we call a function on a string (such as substring) or access it’s properties, the primitive is converted to an object temporarily (which now has methods and properties) and then operation is performed on them.

JavaScript 101: Functions

There are two ways to use functions:

1. Using function declarations:

function print() {

      ...

} 

 

2. Using function expressions:

var print = function() {

    ....

};

The JS parser does work in 2 passes. In 1st pass - it scans and stores all the functions. In 2nd pass, it actually executes the code. This enables us to write function declarations after the function calls in the code.

When the JS parser is in 1st pass, it comes across declaration #1 and creates a reference variable named "print", and stores the address of the function in it. It then moves forward and encounters #2. Since it is not a declaration but a expression, it ignores and moves forward, and starts the 2nd pass.

When the 2nd pass starts, actual code interpretation is carried out. During this, if it comes across a call to a previously declared function, it simply executes the function as it is previously known. However, as it hits #2, it processes the expression in following way:

1. Create a function as per code
2. Create a variable named "print" (as per code)
3. Store the address of the function in the print variable

Basically, the work that is done in 1st pass automatically for declarations, is done in the second pass for #2 type function expressions.

How do they differ?
Time of execution: #1 type declarations are processed before run-time, all functions are available before even starting the interpretation of code. #2 type declarations are processed at run-time, meaning the interpreter creates function and reference on the fly. This also means that you have to create a function expression before calling it. This will not work:

foo(); //first call
var foo = function() {
} // then define

Naming: Both the functions use reference variables to store their address. The reference of #1 types are stored in a variable with the same name as the declaration, and this is automatically done by the interpreter. For #2 types, there is no automatic variable generation, simple a reference is returned. It is up to the programmer to decide what to do with it. It can be left as an anonymous function, or assigned to a variable manually.

Functions are First Class members in JavaScript

First class members in any language are those elements which can be - 
1. Assigned to a variable
2. Passed to a function
3. Returned from a function

All this can be achieved by passing around function references in JavaScript. A classic example is the Array.sort function: 

Array.sort([compareFunction])
compareFunction is a function which knows how to compare any 2 values in the array. The sorting algorithm implicitly calls this function for each sort compare.