MODULE - 4
CHAPTER 5
Handling Data I/O in [Link]
Working with JSON
● O ne of the most common data types that you work with when
implementing [Link] web applications and services isJSON(JavaScript
Object Notation).
● JSON is a lightweight method to convert JavaScript objects into a string
form and then back again.
● This provides an easy method when you need to serialize data objects
when passing them from client to server, process to process, stream to
stream, or when storing them in a database.
hereareseveralreasonstouseJSONtoserializeyourJavaScriptobjects
T
over XML including the following:
● J SON is much more efficient and takes up fewer characters.
Serializing/deserializing JSON is faster than XML because it’s simpler syntax.
● JSON is easier to read from a developer’s perspective because it is similar to
JavaScript syntax.
● TheonlyreasonsyoumightwanttouseXMLoverJSONareforcomplexobjects
or if you have XML/XSLT transforms already in place.
Converting JSON to JavaScript Objects
● A JSON stringrepresentstheJavaScriptobjectinstringform.Thestringsyntax
is similar to code, making it easy to understand. You can use the
[Link](string) method to convert a string that is properly formatted with
JSON into a JavaScript object.
● Forexample,thefollowingcodesnippetdefinesaccountStrasaformattedJSON
string and converts it to a JavaScriptobjectusing[Link]().Thenmember
properties can be accessed via dot notation:
Converting JavaScript Objects to JSON
● N ode also allows you to convert a JavaScript object into a properly
formatted JSON string.
● Thus the string form can be stored in a file or database, sent across an
HTTP connection, or written to a stream/buffer. Use the
[Link](text)methodtoparseJSONtextandgenerateaJavaScript
object:
● For example, the following code defines a JavaScriptobjectthatincludes
string, numeric, and array properties. Using [Link](), it is all
converted to a JSON string:
Using the Buffer Module to Buffer Data
Understanding Buffered Data
● B uffered data is made up of a series of octets in big endian or little
endianformat.Thatmeanstheytakeupconsiderablylessspacethan
textual data.
● Therefore, [Link] provides the Buffer module that gives you the
functionality to create, read, write, and manipulate binary data in a
buffer structure. The Buffer module is global, so you do not needto
use the require() statement to access it.
● Buffered dataisstoredinastructuresimilartothatofanarraybutis
stored outside the normal V8 heap in raw memory allocations.
Therefore a Buffer cannot be resized.
● When convertingbufferstoandfromstrings,youneedtospecifythe
explicit encoding method to be used. Table 5.1 lists the various
encoding methods supported.
Creating Buffers
Writing to Buffers
oucannotextendthesizeofaBufferobjectafterithasbeencreated,butyoucanwrite
Y
data to any location in the buffer. Table 5.2 describes the threemethodsyoucanuse
when writing to buffers.
Reading from Buffers
● T here are several methods for reading from buffers. The simplest is tousethe
toString() method to convert all or part of a buffer to a string.
● However, you can alsoaccessspecificindexesinthebufferdirectlyorbyusing
read().
● Also[Link]providesaStringDecoderobjectthathasawrite(buffer)methodthat
decodes and writes buffered data using the specified encoding. Table 5.3
describes these methods for reading Buffer objects.
Determining Buffer Length
● A common task when dealing with buffers is determining the length, especially
when you create a buffer dynamically from a string.
● The length of a buffer can be determined by calling .length on the Buffer object.
● Todeterminethebytelengththatastringtakesupinabufferyoucannotusethe
.length property.
● Instead you needtouse[Link](string,[encoding]).Notethatthereis
adifferencebetweenthestringlengthandbytelengthofabuffer.Toillustratethis
consider the followings statements:
Copying Buffers
sing the Stream Module to Stream
U
Data
Readable Streams
eadablestreamobjectsalsoprovideanumberoffunctionsthatallowyoutoreadand
R
manipulate them. Table 5.4 lists the methods available on a Readable stream object.
o implementyourowncustomReadablestreamobject,youneedtofirstinherit
T
the functionality forReadablestreams.Thesimplestwaytodothatistousethe
util module’s inherits() method:
var util = require('util');
[Link](MyReadableStream, [Link]);
Then you create an instance of the object call:
[Link](this, opt);
oualsoneedtoimplementa_read()methodthatcallspush()tooutputthedata
Y
from the Readable object. The push() callshouldpusheitheraString,Buffer,or
null.
Writable Streams
● W ritable streams are designedtoprovideamechanismtowritedata
into a form that can easily be consumed in another area of code.
● Some common examples of Writable streams are:
● W ritable streams providethewrite(chunk,[encoding],[callback])
method to write dataintothe stream, wherechunkcontainsthe
datatowrite,encodingspecifiesthestringencodingifnecessary,
and callback specifies a callback function to execute whenthe
data has been fully flushed.
● The write() function returns true if the data was written
successfully. Writable streams also expose the following events:
o implement your own custom Writable stream object, you need to first
T
inheritthefunctionalityforWritablestreams.Thesimplestwaytodothatis
to use the util module’s inherits() method:
var util = require('util');
[Link](MyWritableStream, [Link]);
Then you create an instance of the object call:
stream. [Link](this, opt);
oualsoneedtoimplementa_write(data,encoding,callback)methodthat
Y
stores the data for the Writable object. Listing 5.7 illustrates the basics of
implementing and writing to a Writable stream. Listing 5.7 Output shows
the result.
Duplex Streams
● A Duplex stream combines Readable and Writable functionality. A
good example of a duplex stream is a TCP socket connection. You
can read and write from the socket connection once it has been
created.
● To implement your own custom Duplex stream object, you need to
first inherit the functionality for Duplex streams. The simplestwayto
do that is to use the util module’s inherits() method:
var util = require('util');
[Link](MyDuplexStream, [Link]);
● Then you create an instance of the object call:
stream. [Link](this, opt);
● T heoptparameterwhencreatingaDuplexstreamacceptsanobject
with the property allowHalfOpen set to true or false. If this option is
true, then the readable side stays open even after the writable side
has ended and vice versa. If this option is set to false, ending the
writable side also ends the readable side and vice versa.
● When you implement a Duplex stream,you need to implement
otha_read(size)anda_write(data,encoding,callback)method
b
when prototyping your Duplex class.
Transform Streams
A
● nother type of stream is the Transform stream.
● ATransformstreamextendstheDuplexstreambutmodifiesthedata
between the Writable stream and the Readable stream.
● Thiscanbeusefulwhenyouneedtomodifydatafromonesystemto
another.
● Some examples of Transform streams are
○ zlib streams
○ crypto streams
● A majordifferencebetweentheDuplexandtheTransformstreamsis
that for Transforms you do not need to implement
the _read() and _write() prototype methods.
● T hese are provided as pass-through functions. Instead, you
implement the _transform(chunk, encoding, callback) and
_flush(callback) methods.
● The _transform() method should accept the data from write()
requests, modify it, and then push() out the modified data.
● L isting 5.9 illustrates the basics of implementing a Transform
stream. The stream accepts JSON strings, converts them to
objects,andthenemitsacustomeventnamedobjectthatsends
the object to any listeners.
● The _transform() function also modifies the object to include a
handled property and then sends a string form on. Notice that
lines 18–21 implement the object event handler function that
displays certain attributes.
● In Listing 5.9 Output, notice thatthe JSONstringsnow include
the handled property.
Piping Readable Streams to Writable Streams
● O ne of the coolest things you can do with stream objectsisto
chain Readable streams to Writable streams using the
pipe(writableStream, [options]) function.
● This does exactly what the name implies. The output from the
Readable stream is directly input intothe Writablestream. The
optionsparameteracceptsanobjectwiththeendpropertysetto
true or false.
● Whenend is true,theWritablestreamendswhentheReadable
stream ends. This is the default behavior. For example:
[Link](writeStream, {end:true});
● Y ou can also break the pipe programmatically using the
unpipe(destinationStream) option.
● Listing 5.10 implements a Readable stream and a Writable
stream and then uses the pipe() function to chain them together.
● To showyou thebasicprocess, the datainputfromthe_write()
method is output to the console in Listing 5.10 Output.
ompressingandDecompressingData
C
with Zlib
● N [Link] provides an excellent library in the Zlib module that
allows you to easily and efficiently compress and decompress
data in buffers.
● Keep in mindthat compressingdata takesCPUcycles. Soyou
should becertainofthebenefitsofcompressingthedatabefore
incurring the compression/decompression cost.
● The compression methods supported by Zlib are:
Compressing and Decompressing Buffers
● T he Zlib module provides several helper functions that makeit
easy to compress and decompress data buffers.
● Theseallusethesamebasicformatoffunction(buffer,callback),
where function is the compression/decompression method,
buffer is the buffer to be compressed/decompressed, and
callback is the callback function executed after the
compression/decompression occurs.
Compressing/Decompressing Streams
● C ompressing/decompressing streams using Zlib is slightly
different from compressing/decompressing buffers.
● Instead, you use the pipe() function to pipe the data from one
stream through the compression/decompression object into
another stream.
● This can apply to compressing any Readable streams into
Writable streams.