The Complete Node.

js Study Guide
From JavaScript Foundations to Production-Grade Backend Engineering
Shubhangam Singh · VIT Vellore · [Link] CSE 2023–2027

This guide explains every concept mechanistically — not just what to write, but
why it works. Every code block is annotated. Read it alongside building, not
instead of building.
PART 1

JavaScript Foundations
[Link] is JavaScript running outside the browser. That sentence seems simple, but it means
every subtle JS behaviour — closures, the event loop, the prototype chain — follows you into
your backend code. Shaky JS fundamentals produce confusing Node bugs. This part builds
those concepts as bedrock before you write a single line of server code.

1.1 Variables and Scoping

JavaScript has three variable declarations: var, let, and const. The difference is not about
memorising a rule — it is about understanding where a variable lives in memory relative to the
code structure around it.

Understanding var: hoisted and function-scoped

var is function-scoped, meaning it ignores block boundaries like if-statements and for-loops. It is
also hoisted — the declaration is moved to the top of its containing function at parse time, but
the assignment is not. This produces the classic surprise of getting undefined instead of an error
when accessing a variable before its assignment line.
// var is FUNCTION-scoped — does NOT respect block boundaries
function exampleVar() {
[Link](x); // undefined (NOT an error — declaration is hoisted)
var x = 10;

if (true) {
var x = 99; // This modifies the SAME x — var ignores the block
}
[Link](x); // 99 — this is the classic var bug
}

Understanding let: block-scoped with the Temporal Dead Zone

let is block-scoped, meaning any pair of curly braces creates a new scope. It is also hoisted, but
into a "Temporal Dead Zone" — a region where the variable exists in memory but is not yet
initialised. Accessing it in the TDZ throws a ReferenceError instead of silently returning
undefined, which makes bugs far easier to diagnose.
function exampleLet() {
// [Link](y); // ReferenceError: Cannot access 'y' before initialisation
let y = 10;

if (true) {
let y = 99; // SEPARATE y, local to this block
[Link](y); // 99
}
[Link](y); // 10 — outer y untouched
}
Understanding const: immutable binding, not immutable value
const prevents the variable binding from being reassigned, but it does not make objects or
arrays immutable. You can still mutate the properties of a const object — you just cannot point
the variable to a completely different object.
const config = { port: 3000 };
[Link] = 4000; // OK — mutating the object, not rebinding the variable
// config = {}; // TypeError: Assignment to constant variable

// Rule of thumb: use const by default, let when the value must change,
// never use var.
💡 In [Link] files, use const for everything unless you have a specific reason to reassign. This
makes your intent clear and prevents accidental reassignments.

1.2 Functions and Closures

A closure is one of the most important concepts in JavaScript and the backbone of most [Link]
patterns — middleware chains, callbacks, factory functions, and module encapsulation all rely
on closures. Understanding them deeply will unlock your ability to read Express source code,
npm packages, and framework internals.

The three ways to define a function

// 1. Declaration — hoisted entirely, callable before the line it appears on
function greet(name) {
return `Hello, ${name}`;
}

// 2. Expression — only the variable declaration is hoisted, not the function

const greetExpr = function(name) {
return `Hello, ${name}`;
};

// 3. Arrow function — compact AND inherits 'this' from surrounding scope

// (why this matters becomes clear in the next section)
const greetArrow = (name) => `Hello, ${name}`;

What a closure actually is

A closure is formed when a function is defined inside another function and the inner function
references a variable from the outer function's scope. Even after the outer function has returned
and its call stack frame is gone, the inner function retains a live reference to that variable in
memory. The variable is not garbage-collected because something still holds a reference to it.
function makeCounter(startValue) {
// 'count' lives in makeCounter's scope
let count = startValue;

// The returned object has two methods, both closures over the SAME 'count'
return {
increment() { count++; },
decrement() { count--; },
 value() { return count; }
};
}

const counter = makeCounter(10);

[Link](); // count = 11
[Link](); // count = 12
[Link](); // count = 11
[Link]([Link]()); // 11

Closures in [Link] middleware (the practical payoff)

Express middleware is built entirely on closures. A middleware factory function creates a
preconfigured middleware by capturing configuration in its closure. Every incoming request runs
the inner function, all sharing the same closed-over state.
// A rate-limiter middleware factory
function rateLimiter(maxRequests) {
const requests = new Map(); // shared across ALL requests via closure

return function(req, res, next) {

const ip = [Link];
const count = [Link](ip) || 0;

if (count >= maxRequests) {

return [Link](429).json({ error: 'Too many requests' });
}

[Link](ip, count + 1);

next();
};
}

// The Map is created once when rateLimiter(100) is called.

// Every request runs the inner function and shares it.
[Link](rateLimiter(100));

1.3 The "this" Keyword

The this keyword is the single most misunderstood concept in JavaScript. Its value is not
determined by where a function is defined. It is determined by how and where the function is
called at runtime. This distinction is critical in [Link] because Express, event emitters, and
class-based patterns all manipulate this.

The core rule: this equals the object to the left of the dot at call time
const user = {
name: 'Shubhangam',
greet() {
[Link](`Hello from ${[Link]}`); // this = user
}
};
 [Link](); // "Hello from Shubhangam"

// Detaching loses context

const greetFn = [Link];
greetFn(); // "Hello from undefined" — no object to the left of the call

Arrow functions and lexical this

Arrow functions do not have their own this. They inherit this from the enclosing lexical scope —
the scope where they were written, not where they are called. This makes them ideal for
callbacks inside class methods.
class Server {
constructor() {
[Link] = 3000;
}

start() {
// Regular function: 'this' inside setTimeout would be undefined (strict mode)
// Arrow function: inherits 'this' from start(), which is the Server instance
setTimeout(() => {
[Link](`Server running on ${[Link]}`); // 3000 ✅
}, 1000);
}
}

// Explicit binding with call, apply, bind

function introduce(greeting) {
[Link](`${greeting}, I'm ${[Link]}`);
}
const dev = { name: 'Shubhangam' };
[Link](dev, 'Hi'); // call — passes args individually
[Link](dev, ['Hello']); // apply — passes args as array
const bound = [Link](dev); // bind — returns a new permanently bound
function
bound('Hey');

1.4 Prototypes and Classes

Every object in JavaScript has a hidden [[Prototype]] link pointing to another object. When you
access a property that does not exist on the object itself, JavaScript walks up this prototype
chain until it finds it or hits null. ES6 classes are syntactic sugar over this prototype system —
understanding the underlying mechanism explains every "class" behaviour you will encounter in
Node packages.
// The underlying prototype mechanism
const animal = {
breathe() { return 'breathing...'; }
};

const dog = [Link](animal); // dog.__proto__ === animal

[Link] = function() { return 'woof!'; };
 [Link]([Link]()); // own property — found immediately
[Link]([Link]()); // not on dog → found on animal via chain

// ES6 Classes (preferred syntax — same mechanism underneath)

class EventEmitter {
constructor() {
this._events = {};
}

on(event, listener) {
if (!this._events[event]) this._events[event] = [];
this._events[event].push(listener);
return this; // enables chaining: [Link]('a', fn).on('b', fn)
}

emit(event, ...args) {
(this._events[event] || []).forEach(fn => fn(...args));
}
}

// extends wires up prototype chain: [Link] → [Link]

class MyEmitter extends EventEmitter {
doWork() {
[Link]('done', { result: 42 }); // inherited from EventEmitter
}
}

const e = new MyEmitter();

[Link]('done', data => [Link]('Result:', [Link])); // 42
[Link]();

1.5 Destructuring, Spread, and Rest

These features appear everywhere in modern Express and Node code. Fluency with them
makes reading framework source code and npm packages feel natural rather than cryptic.
// Object destructuring with renaming and defaults
const req = {
body: { name: 'Shubhangam', email: 'dev@[Link]' },
params: { id: '42' },
query: { page: '2', limit: '10' }
};

const { body: { name, email }, params: { id } } = req;

const { query: { page = 1, limit = 10 } } = req; // default values

// Array destructuring
const [first, second, ...rest] = [10, 20, 30, 40, 50];
// first = 10, second = 20, rest = [30, 40, 50]

// Rest parameters — collect remaining function args into an array

function sum(...numbers) {
 return [Link]((acc, n) => acc + n, 0);
}
[Link](sum(1, 2, 3, 4, 5)); // 15

// Spread — expand an iterable into individual elements

const baseConfig = { host: 'localhost', port: 3000, debug: false };
const envConfig = { port: 5000, debug: true }; // overrides

// Later keys overwrite earlier keys — very common for config merging
const finalConfig = { ...baseConfig, ...envConfig };
// { host: 'localhost', port: 5000, debug: true }

1.6 Arrays and Functional Patterns

The map, filter, reduce, find, and flat methods are the tools you reach for every time you
transform data in a Node API. Understanding them as higher-order functions — functions that
accept other functions as arguments — unlocks composable, readable data transformation.
const posts = [
{ id: 1, title: 'Node Basics', views: 1200, published: true },
{ id: 2, title: 'Express Guide', views: 800, published: false },
{ id: 3, title: 'MongoDB Mastery', views: 2100, published: true },
];

// map — transform every element, returns same-length array

const titles = [Link](p => [Link]);

// filter — keep elements passing a test

const published = [Link](p => [Link]);

// reduce — fold all elements into a single value (most flexible)

const totalViews = [Link]((acc, p) => acc + [Link], 0); // 4100

// find — returns the FIRST match (or undefined)

const popular = [Link](p => [Link] > 1000);

// Chaining — filter then map in one expression

const publishedTitles = posts
.filter(p => [Link])
.map(p => [Link]());

// flat and Set — flatten nested arrays and deduplicate

const tagGroups = [['node', 'backend'], ['express', 'node'], ['mongo']];
const unique = [...new Set([Link]())];
// ['node', 'backend', 'express', 'mongo']

1.7 Asynchronous JavaScript — Callbacks → Promises →

Async/Await
This is the most important section for [Link]. Node's entire superpower — handling thousands
of concurrent connections on a single thread — rests on asynchronous programming. You must
understand all three levels, not just async/await, because older npm packages and Node's own
core modules use callbacks internally.

Level 1: Callbacks
A callback is a function you pass as an argument, saying "call this when you are done." Node
uses the error-first convention: the first argument is always an error (or null if successful), and
the second argument is the result.
const fs = require('fs');

[Link]('./[Link]', 'utf8', function(err, data) {

// err is null on success, an Error object on failure
if (err) {
[Link]('Failed:', [Link]);
return; // ALWAYS return after handling error — never let code fall through
}
[Link]('Contents:', data);
});

// This line prints BEFORE the file is read.

// Node does NOT wait — it queues the read and moves on.
[Link]('This runs first!');

// Callback hell: when operations depend on each other, nesting gets ugly
[Link]('./[Link]', 'utf8', (err, userId) => {
if (err) return handleError(err);
[Link](userId, (err, user) => {
if (err) return handleError(err);
[Link]([Link], (err, posts) => {
if (err) return handleError(err);
// Three levels already. Imagine ten. This is "callback hell."
});
});
});

Level 2: Promises
A Promise represents a value that will be available in the future. It has three states: pending
(waiting), fulfilled (success), or rejected (failure). Once it settles, its state never changes.
Promises can be chained, which eliminates callback nesting.
// Creating a Promise
const readFilePromise = new Promise((resolve, reject) => {
[Link]('./[Link]', 'utf8', (err, data) => {
if (err) reject(err); // transitions to rejected state
else resolve(data); // transitions to fulfilled state
});
});

// Consuming a Promise via chaining

readFilePromise
 .then(data => [Link](data)) // .then runs on fulfil
.then(obj => [Link](obj))
.catch(err => [Link](err)) // ONE catch handles any failure in chain
.finally(() => [Link]('Done')); // runs regardless

// Running multiple promises in parallel with [Link]

const [user, posts, config] = await [Link]([
fetchUser(id), // all three start simultaneously
fetchPosts(id),
fetchConfig()
]);
// Resolves when ALL three complete; rejects if ANY one fails

Level 3: Async/Await
Async/await is syntax sugar over Promises. It makes asynchronous code read like synchronous
code, which dramatically improves readability. An async function always returns a Promise.
await pauses execution of the current async function — not the entire Node process — until the
awaited Promise settles.
async function getUserWithPosts(userId) {
try {
const user = await fetchUser(userId); // pauses here until done
const posts = await fetchPosts([Link]); // then starts this
return { user, posts };
} catch (err) {
// Any rejection anywhere in the try block lands here
[Link]('Failed:', [Link]);
throw err; // re-throw so the caller knows it failed
}
}

// IMPORTANT: sequential awaits are slow when operations are independent

async function getAll(userId) {
// ❌ Sequential — each waits for the previous before starting:
// const user = await fetchUser(userId);
// const posts = await fetchPosts(userId);

// ✅ Parallel — both fire immediately, then wait for both:

const [user, posts] = await [Link]([fetchUser(userId), fetchPosts(userId)]);
return { user, posts };
}
💡 A very common interview question: "What is the difference between async/await and Promises?"
The correct answer is: async/await IS Promises — it is just different syntax. Under the hood, an
async function returns a Promise and await is .then() in disguise.

1.8 The Event Loop — Deep Dive

The event loop is what makes [Link] special. It is the mechanism that allows Node to handle
tens of thousands of concurrent connections on a single thread. Understanding it is the
difference between writing correct async code and writing code with subtle timing bugs that only
appear under load.

The mental model

Imagine a chef who cannot multitask. The chef can only cook one thing at a time. But instead of
standing at the stove waiting for water to boil, the chef puts the pot on and goes to do other
tasks. When the water boils (the I/O is ready), the chef is notified and comes back to finish that
dish. [Link] is that chef. The event loop is the system by which the chef knows what to do
next.

The six phases of the event loop

EVENT LOOP PHASE ORDER (each iteration = one "tick"):

1. TIMERS — runs expired setTimeout and setInterval callbacks

2. PENDING I/O — runs I/O error callbacks from previous iteration
3. IDLE/PREPARE — internal Node use only
4. POLL — retrieves new I/O events; blocks here if queue empty
5. CHECK — runs setImmediate callbacks
6. CLOSE CALLBACKS — [Link]('close', ...) and similar cleanup

Between EVERY phase (highest to lowest priority):

[Link] queue → runs ALL nextTick callbacks
Promise microtask queue → runs ALL resolved promise callbacks

// Demonstrating event loop ordering — trace this carefully

[Link]('1 — synchronous');

setTimeout(() => [Link]('5 — setTimeout (Timers phase)'), 0);

setImmediate(() => [Link]('4 — setImmediate (Check phase)'));
[Link]().then(() => [Link]('3 — [Link] (microtask)'));
[Link](() => [Link]('2 — nextTick (inter-phase queue)'));

[Link]('1b — synchronous end');

// Output: 1, 1b, 2, 3, 4, 5
// Explanation:
// Sync code runs first (1, 1b).
// nextTick queue drains before any phase (2).
// Promise microtask queue drains next (3).
// Event loop starts — Check phase runs setImmediate (4).
// Timers phase runs setTimeout (5).

The critical implication: never block the event loop

If you do CPU-heavy work synchronously — sorting a million records, parsing a huge JSON
string — the event loop is blocked for that entire duration. No incoming HTTP requests can be
processed, no database callbacks can run, no timers can fire. This is the single most common
[Link] performance mistake.
// BAD — sorting 1 million items blocks the event loop
[Link]('/sorted', (req, res) => {
 const sorted = [Link]((a, b) => a - b); // sync, blocks entire server
[Link](sorted);
});

// GOOD — offload to a worker thread (covered in Part 3)

[Link]('/sorted', async (req, res) => {
const sorted = await sortInWorkerThread(massiveArray);
[Link](sorted);
});

1.9 ES Modules vs CommonJS

Node originally used CommonJS (require/[Link]). ES Modules (import/export) became
the official JavaScript standard and are now fully supported in modern Node. Both coexist in the
ecosystem — you will encounter both, and sometimes need to use both in the same project.
// ── CommonJS (traditional Node) ──────────────────────────────────────────
// [Link]
function add(a, b) { return a + b; }
function subtract(a, b) { return a - b; }
[Link] = { add, subtract };

// [Link]
const { add, subtract } = require('./mathUtils');
const path = require('path'); // built-in module
const express = require('express'); // npm package
// require() is SYNCHRONOUS. It runs the module once and CACHES the result.
// All subsequent require() calls for the same file return the cached export.

// ── ES Modules (modern standard) ─────────────────────────────────────────

// [Link] (or set "type": "module" in [Link])
export function add(a, b) { return a + b; }
export function subtract(a, b) { return a - b; }
export default class Calculator { /* ... */ }

// [Link]
import Calculator, { add, subtract } from './[Link]';
import path from 'path';

// In ESM, __dirname and __filename are not defined by default:

import { fileURLToPath } from 'url';
import { dirname } from 'path';
const __filename = fileURLToPath([Link]);
const __dirname = dirname(__filename);
💡 For new projects, set "type": "module" in [Link] and use ESM. For learning alongside older
tutorials and npm packages, CommonJS is simpler. The concepts are identical — only the syntax
differs.
PART 2

[Link] Architecture
With the JavaScript foundations locked in, we can now explore what [Link] actually is and how
it works under the hood. This knowledge is what separates developers who just use Node from
developers who understand why their code behaves a certain way.

2.1 What [Link] Actually Is (V8 + libuv)

[Link] is not a language. It is a runtime environment built by combining two powerful pieces of
software:

• V8 — Google's open-source JavaScript engine (the same one inside Chrome). V8

compiles JavaScript directly to native machine code using Just-In-Time compilation. It
handles all JavaScript execution — variable storage, function calls, garbage collection.
• libuv — A C library that provides the event loop, a thread pool for offloading blocking
operations, and cross-platform async I/O primitives. libuv is the engine under [Link],
[Link], crypto.pbkdf2, and every other non-blocking operation.

When you call [Link], here is the complete journey:

Your JS code
↓ calls
Node C++ binding (bridges JS to C)
↓ submits work to
libuv (queues disk read with the OS via epoll/kqueue/IOCP)
↓ Node event loop continues — handles other requests
↓ OS signals libuv when disk read is complete
libuv places your callback in the poll queue
↓
Event loop picks it up → your callback runs
This is why Node can handle 10,000 concurrent HTTP connections with one thread — it
delegates I/O to the OS and loops over ready callbacks rather than blocking on each operation.

2.2 The Node Process and Globals

The process object represents the running Node process and gives you access to environment
variables, CLI arguments, signals, and lifecycle events.
[Link]; // 'v22.x.x'
[Link]; // 'linux', 'darwin', 'win32'
[Link]; // environment variables
[Link]; // ['node', '[Link]', '--flag', 'value']
[Link](); // current working directory
[Link]; // process ID — useful in distributed logging

// Graceful shutdown — ALWAYS handle in production

 [Link]('SIGTERM', async () => {
// Docker/Kubernetes sends SIGTERM before killing a container
[Link]('Shutting down gracefully...');
await [Link](); // stop accepting new connections
await [Link](); // close DB connections cleanly
[Link](0);
});

[Link]('uncaughtException', (err) => {

// Unhandled synchronous error — process is in unknown state.
// Log thoroughly, then ALWAYS exit.
[Link]('UNCAUGHT EXCEPTION:', err);
[Link](1);
});

[Link]('unhandledRejection', (reason) => {

// Promise rejected with no .catch() — crashes process in Node 15+
[Link]('UNHANDLED REJECTION:', reason);
[Link](1);
});

2.3 npm and [Link]

npm (Node Package Manager) is the world's largest software registry. [Link] is your
project's manifest — it declares dependencies, scripts, and metadata.
{
"name": "my-api",
"version": "1.0.0",
"type": "commonjs",
"main": "src/[Link]",
"scripts": {
"start": "node src/[Link]",
"dev": "nodemon src/[Link]",
"test": "jest --coverage"
},
"dependencies": {
"express": "^4.18.2", // ^ = accept minor+patch updates (4.x.x)
"mongoose": "~7.6.3", // ~ = accept only patch updates (7.6.x)
"jsonwebtoken": "9.0.0" // no prefix = exact version only
},
"devDependencies": {
"nodemon": "^3.0.1", // only needed during development
"jest": "^29.0.0"
}
}
# Key npm commands
npm init -y # create [Link] with defaults
npm install express # add to dependencies
npm install nodemon --save-dev # add to devDependencies

npm run dev # run the dev script

npm audit # check for known security vulnerabilities
npm audit fix # auto-fix safe vulnerabilities

# [Link]: records exact installed versions.

# ALWAYS commit this. Ensures reproducible installs across machines.
# node_modules/: NEVER commit. Add to .gitignore.
PART 3

[Link] Core Modules

Core modules are built into Node — no installation needed. Understanding them deeply makes
you a far better backend engineer because every framework (Express, Fastify, NestJS) is built
on top of these primitives.

3.1 path, os, and url

const path = require('path');
const os = require('os');
const url = require('url');

// ── path ──────────────────────────────────────────────────────────────────
// ALWAYS use [Link] instead of string concatenation.
// String concat breaks on Windows which uses \ instead of /
const filePath = [Link](__dirname, 'uploads', '[Link]');

[Link]('/home/user/[Link]'); // '[Link]'
[Link]('/home/user/[Link]'); // '/home/user'
[Link]('/home/user/[Link]'); // '.txt'
[Link]('uploads', '[Link]'); // absolute path from cwd

// ── os ────────────────────────────────────────────────────────────────────
[Link]().length; // number of CPU cores — used for [Link]() sizing
[Link](); // total RAM in bytes
[Link](); // available RAM in bytes
[Link](); // temp dir — safe place for temporary files

// ── url ───────────────────────────────────────────────────────────────────
const myUrl = new URL('[Link]
[Link]; // 'https:'
[Link]; // '[Link]'
[Link]; // '/users'
[Link]('page'); // '2'
[Link]('limit'); // '10'

3.2 fs — File System

The fs module provides everything you need to interact with the file system. The key principle:
never use synchronous (blocking) fs methods inside request handlers. Only use them at startup
for one-time config loading.
const fsPromises = require('fs').promises; // Promise-based API (preferred)
const fs = require('fs');

// ── Sync (blocking) — ONLY use at startup, never inside request handlers ──

const config = [Link]([Link]('./[Link]', 'utf8'));
 // This blocks the event loop. OK once at startup; catastrophic inside a handler.

// ── Async Promises (preferred) ────────────────────────────────────────────

async function readConfig() {
try {
const data = await [Link]('./[Link]', 'utf8');
return [Link](data);
} catch (err) {
if ([Link] === 'ENOENT') return {}; // ENOENT = file not found
throw err;
}
}

// Writing, appending, directories

await [Link]('./[Link]', 'Hello
', 'utf8'); // overwrites
await [Link]('./[Link]', `${new Date().toISOString()} event
`);
await [Link]('./uploads/images', { recursive: true }); // like mkdir -p
await [Link]('./fullDir', { recursive: true }); // delete tree

// File metadata
const stat = await [Link]('./[Link]');
[Link]; // size in bytes
[Link]; // last modified Date
[Link](); // true
[Link](); // false

3.3 events — EventEmitter

The EventEmitter is the backbone of Node's asynchronous event-driven architecture. HTTP
servers, streams, database connections — all extend EventEmitter. It implements the Observer
pattern: objects emit named events, and listeners subscribed to those events are notified and
run.
const EventEmitter = require('events');

class OrderService extends EventEmitter {

constructor(db) {
super();
[Link] = db;
}

async createOrder(orderData) {
const order = await [Link](orderData);

// The core logic does NOT know or care what happens after.
// It just announces the fact. Listeners are decoupled.
[Link]('order:created', order);

return order;
}
 }

const orderService = new OrderService(db);

// Completely decoupled side effects — add/remove without touching createOrder

[Link]('order:created', order => sendConfirmationEmail([Link]));
[Link]('order:created', order => notifyWarehouse(order));
[Link]('order:created', order => logAnalytics(order));

// Key methods
[Link]('event', listener); // persistent listener
[Link]('event', listener); // fires exactly once, then auto-removed
[Link]('event', listener); // remove specific listener
[Link]('event', ...args); // trigger all listeners synchronously
[Link]('event'); // how many listeners attached

// CRITICAL: always handle the 'error' event.

// If 'error' is emitted with no listener, Node throws and crashes the process.
[Link]('error', err => [Link]('Emitter error:', [Link]));

3.4 stream — Streams and Backpressure

Streams let you process data in chunks rather than loading everything into memory at once.
When handling large files (logs, CSV imports, video uploads), streams are not optional — they
are the difference between an API that works and one that crashes under load.
const fs = require('fs');
const zlib = require('zlib');
const { pipeline } = require('stream');
const { promisify } = require('util');
const pipelineAsync = promisify(pipeline);

// WITHOUT streams: loading a 2GB file crashes your server

// const data = [Link]('[Link]'); // 2GB into RAM

// WITH streams: memory usage stays constant regardless of file size

const readStream = [Link]('[Link]', {
highWaterMark: 64 * 1024 // process in 64KB chunks
});

[Link]('data', chunk => processChunk([Link]()));

[Link]('end', () => [Link]('Done'));
[Link]('error', err => [Link](err));

// Four stream types:

// Readable — source ([Link], [Link])
// Writable — destination ([Link], [Link])
// Duplex — both Readable and Writable ([Link])
// Transform — Duplex that modifies data as it passes ([Link])

// pipe() connects streams. Use pipeline() in production — handles errors.

async function compressFile(input, output) {
 await pipelineAsync(
[Link](input),
[Link](), // Transform: compress data in flight
[Link](output) // Writable: save compressed data
);
// If any stream errors, ALL streams are destroyed — no resource leaks
}
💡 Backpressure: when your readable produces data faster than the writable can consume it,
[Link]() returns false. You must pause the readable and resume it when the writable drains.
pipe() and pipeline() handle this automatically — which is why you should always use them instead
of manually piping data/end events.

3.5 http — Raw HTTP Server

Understanding the raw http module shows you exactly what Express wraps and abstracts.
Every Express feature — request body parsing, routing, response helpers — corresponds to
something in this raw interface.
const http = require('http');

const server = [Link](async (req, res) => {

// req = [Link] (Readable stream + headers + method + url)
// res = [Link] (Writable stream + response helpers)

// Reading the body (it arrives as a stream — collect chunks)

if ([Link] === 'POST') {
let body = '';
[Link]('data', chunk => { body += [Link](); });
[Link]('end', () => {
try {
const parsed = [Link](body);
[Link](201, { 'Content-Type': 'application/json' });
[Link]([Link]({ created: true, data: parsed }));
} catch {
[Link](400, { 'Content-Type': 'application/json' });
[Link]([Link]({ error: 'Invalid JSON' }));
}
});
return;
}

// Simple manual routing (this is exactly what Express automates)

const { pathname } = new URL([Link], `[Link]

if (pathname === '/health') {

[Link](200, { 'Content-Type': 'application/json' });
[Link]([Link]({ status: 'ok', uptime: [Link]() }));
} else {
[Link](404, { 'Content-Type': 'application/json' });
[Link]([Link]({ error: 'Not found' }));
}
});
 [Link](3000, () => [Link]('Listening on port 3000'));

3.6 crypto
const crypto = require('crypto');

// Hashing (one-way — cannot be reversed)

const hash = [Link]('sha256')
.update('data to hash')
.digest('hex');

// Secure random tokens (for password resets, API keys)

const token = [Link](32).toString('hex'); // 64-char hex string

// HMAC — keyed hash for signing data (what JWT does internally)
const sig = [Link]('sha256', 'secret-key')
.update('message')
.digest('hex');

// AES-256-GCM encryption
const key = [Link](32); // 256-bit key
const iv = [Link](16); // initialisation vector

function encrypt(plaintext) {
const cipher = [Link]('aes-256-gcm', key, iv);
const enc = [Link]([[Link](plaintext, 'utf8'), [Link]()]);
return { data: [Link]('hex'), tag: [Link]().toString('hex') };
}

function decrypt({ data, tag }) {

const decipher = [Link]('aes-256-gcm', key, iv);
[Link]([Link](tag, 'hex'));
return [Link]([
[Link]([Link](data, 'hex')),
[Link]()
]).toString('utf8');
}

3.7 worker_threads and cluster

These two modules solve the same problem (using multiple CPU cores) but in different ways.
worker_threads shares memory between threads in the same process — good for CPU-
intensive computations. cluster creates separate OS processes — good for maximising HTTP
throughput.
const { Worker, isMainThread, parentPort, workerData } = require('worker_threads');

// worker_threads: offload CPU-heavy work without blocking the event loop

if (isMainThread) {
const worker = new Worker(__filename, {
 workerData: { items: [5, 3, 1, 4, 2] }
});
[Link]('message', sorted => [Link]('Sorted:', sorted));
} else {
const sorted = [Link]((a, b) => a - b);
[Link](sorted);
}

// cluster: fork one worker per CPU core to handle HTTP requests in parallel
const cluster = require('cluster');
const os = require('os');

if ([Link]) {
for (let i = 0; i < [Link]().length; i++) [Link]();
[Link]('exit', () => [Link]()); // auto-restart crashed workers
} else {
require('./app').listen(3000); // each worker runs the full server
[Link](`Worker ${[Link]} started`);
}
PART 4

[Link] — Building REST APIs

Express is a minimal, unopinionated web framework. Its power comes from its middleware
pipeline — a linear chain of functions through which every request passes in order.
Understanding the pipeline architecture is the key to understanding everything Express does.

4.1 Setup and the Middleware Pipeline

A middleware is simply a function with the signature (req, res, next). It receives the request, can
read or modify it, and must either call next() to pass control to the next middleware, or send a
response to end the cycle. If it does neither, the request hangs indefinitely.
const express = require('express');
const morgan = require('morgan'); // HTTP request logging
const helmet = require('helmet'); // security headers
const cors = require('cors'); // cross-origin resource sharing

const app = express();

// Order matters — middleware runs top to bottom for every request

[Link](helmet()); // 1. Security headers — always first

[Link](cors({
origin: [Link].CLIENT_URL,
credentials: true // allow cookies across origins
}));

[Link](morgan('dev')); // 2. Request logging: GET /users 200 45ms

[Link]([Link]({ limit: '10mb' })); // 3. Parse JSON body

[Link]([Link]({ extended: true })); // Parse form data

[Link]('/api/users', require('./routes/user')); // 4. Route handlers

[Link]('/api/posts', require('./routes/post'));

// 5. 404 catcher — any request that fell through all routes

[Link]((req, res) => {
[Link](404).json({ error: `${[Link]} ${[Link]} not found` });
});

// 6. Global error handler — MUST have 4 params, Express detects it

[Link]((err, req, res, next) => {
const status = [Link] || 500;
[Link](status).json({ error: [Link] || 'Internal Server Error' });
});

[Link]([Link] || 5000);
4.2 Routing, Controllers, and Services
A well-structured Express codebase separates three concerns: routes define the URL and
HTTP method, controllers handle the HTTP layer (reading req, writing res), and services contain
the business logic. This separation makes code testable, reusable, and easy to reason about.
// routes/[Link]
const router = require('express').Router();
const userCtrl = require('../controllers/[Link]');
const { protect, restrictTo } = require('../middleware/auth');
const validate = require('../middleware/validate');
const { createUserSchema } = require('../validators/user');

[Link]('/', [Link]);
[Link]('/', validate(createUserSchema), [Link]);
[Link]('/:id', [Link]); // :id → [Link]
[Link]('/:id', protect, [Link]);
[Link]('/:id', protect, restrictTo('admin'), [Link]);

[Link] = router;

// controllers/[Link] — handles HTTP layer only

const UserService = require('../services/[Link]');

[Link] = async (req, res, next) => {

try {
const { page = 1, limit = 10, sort = 'createdAt' } = [Link];
const result = await [Link]({ page: +page, limit: +limit, sort });

[Link](200).json({
success: true,
data: [Link],
pagination: { total: [Link], page: +page }
});
} catch (err) {
next(err); // passes to global error handler — never crash, always delegate
}
};

[Link] = async (req, res, next) => {

try {
const user = await [Link]([Link]);
if (!user) {
const err = new Error('User not found');
[Link] = 404;
return next(err);
}
[Link](200).json({ success: true, data: user });
} catch (err) {
next(err);
}
};
4.3 Auth Middleware and Request Validation
// middleware/[Link]
const jwt = require('jsonwebtoken');
const User = require('../models/[Link]');

[Link] = async (req, res, next) => {

try {
const header = [Link];
if (!header?.startsWith('Bearer ')) {
const err = new Error('No token provided');
[Link] = 401;
return next(err);
}

const token = [Link](' ')[1];

const decoded = [Link](token, [Link].JWT_SECRET); // throws if expired

[Link] = await [Link]([Link]).select('-password');

if (![Link]) {
const err = new Error('User no longer exists');
[Link] = 401;
return next(err);
}

next(); // authenticated — pass to next handler

} catch (err) {
if ([Link] === 'JsonWebTokenError') [Link] = 401;
if ([Link] === 'TokenExpiredError') [Link] = 401;
next(err);
}
};

[Link] = (...roles) => (req, res, next) => {

// Closure: captures 'roles'. Returns a middleware.
if () {
const err = new Error('Insufficient permissions');
[Link] = 403;
return next(err);
}
next();
};

// middleware/[Link] — Zod schema validation

const { z } = require('zod');

[Link] = (schema) => (req, res, next) => {

try {
[Link]({ body: [Link], params: [Link], query: [Link] });
next();
} catch (err) {
if (err instanceof [Link]) {
return [Link](400).json({
error: 'Validation failed',
 details: [Link](e => ({ field: [Link]('.'), message:
[Link] }))
});
}
next(err);
}
};
PART 5

Databases — Mongoose and MongoDB

5.1 Connecting to MongoDB

const mongoose = require('mongoose');

const connectDB = async () => {

try {
const conn = await [Link]([Link].MONGODB_URI);
[Link](`MongoDB connected: ${[Link]}`);

[Link]('error', err => [Link]('DB error:', err));

[Link]('disconnected', () => [Link]('DB disconnected'));
} catch (err) {
[Link]('DB connection failed:', [Link]);
[Link](1); // cannot run without database
}
};

[Link] = connectDB;

5.2 Schemas, Models, and CRUD

A Mongoose schema defines the shape and validation rules of your documents. A model is the
class you use to query and create documents. The pre-save hook and instance methods are
two of Mongoose's most powerful features.
const mongoose = require('mongoose');
const bcrypt = require('bcryptjs');

const userSchema = new [Link]({

name: {
type: String, required: [true, 'Name is required'], trim: true, maxlength: 50
},
email: {
type: String, required: true, unique: true, lowercase: true,
match: [/^\S+@\S+\.\S+$/, 'Invalid email']
},
password: {
type: String, required: true, minlength: 8,
select: false // excluded from query results by default — never leak passwords
},
role: { type: String, enum: ['user', 'admin'], default: 'user' },
posts: [{ type: [Link], ref: 'Post' }],
isActive: { type: Boolean, default: true }
}, {
timestamps: true // auto-adds createdAt, updatedAt
 });

// Pre-save hook: hash password before storing

[Link]('save', async function(next) {
if () return next(); // skip if password unchanged
[Link] = await [Link]([Link], 12);
next();
});

// Instance method: compare plain password with hash

[Link] = async function(candidate) {
return [Link](candidate, [Link]);
};

const User = [Link]('User', userSchema);

// ── CRUD ──────────────────────────────────────────────────────────────────

// Create
const user = await [Link]({ name, email, password }); // triggers pre-save

// Read
const found = await [Link](id).select('-password -__v');
const users = await [Link]({ isActive: true })
.sort({ createdAt: -1 })
.skip((page - 1) * limit)
.limit(limit)
.lean(); // returns plain JS objects — faster

// Update
const updated = await [Link](
id,
{ $set: { name: 'New Name' } },
{ new: true, runValidators: true } // return updated doc; run validators
);

// Delete (soft delete preferred in production)

await [Link](id, { isActive: false });

5.3 Aggregation Pipeline

Aggregation is MongoDB's most powerful feature. It processes documents through a sequence
of stages, each transforming the data, like an assembly line. Understanding aggregation makes
the difference between one efficient query and ten slow ones.
// Get top 5 authors by published post count, with average views per post
const topAuthors = await [Link]([
// Stage 1: Filter
{ $match: { published: true } },

// Stage 2: Group by author, compute stats

{ $group: {
 _id: '$author', // group key
postCount: { $sum: 1 }, // count docs in group
avgViews: { $avg: '$views' } // average views
}},

// Stage 3: Sort by post count, descending

{ $sort: { postCount: -1 } },

// Stage 4: Top 5 only

{ $limit: 5 },

// Stage 5: Join with Users collection (SQL equivalent: LEFT JOIN)

{ $lookup: {
from: 'users', // target collection
localField: '_id', // field in Post (grouped _id = author ObjectId)
foreignField: '_id', // field in User
as: 'authorInfo' // output array field name
}},

// Stage 6: Flatten the authorInfo array (lookup returns array)

{ $unwind: '$authorInfo' },

// Stage 7: Shape the final output

{ $project: {
_id: 0,
author: '$[Link]',
postCount: 1,
avgViews: { $round: ['$avgViews', 1] }
}}
]);
PART 6

Authentication and Security

6.1 JWT Authentication — Signup, Login, Refresh

The access token + refresh token pattern is the production standard. Access tokens are short-
lived (15 minutes) and stored in memory by the client. Refresh tokens are long-lived (7 days)
and stored in httpOnly cookies that JavaScript cannot read. This combination protects against
both CSRF and XSS attacks.
const jwt = require('jsonwebtoken');
const User = require('../models/[Link]');

const generateTokens = (userId) => {

const accessToken = [Link](
{ id: userId }, [Link].JWT_SECRET,
{ expiresIn: '15m' } // short-lived — 15 minutes
);
const refreshToken = [Link](
{ id: userId }, [Link].JWT_REFRESH_SECRET,
{ expiresIn: '7d' } // long-lived — 7 days
);
return { accessToken, refreshToken };
};

[Link] = async (req, res, next) => {

try {
const { name, email, password } = [Link];

if (await [Link]({ email })) {

const err = new Error('Email already registered'); [Link] = 409;
return next(err);
}

const user = await [Link]({ name, email, password }); // password hashed
via pre-save
const { accessToken, refreshToken } = generateTokens(user._id);

// Refresh token in httpOnly cookie — cannot be read by JavaScript (XSS safe)

[Link]('refreshToken', refreshToken, {
httpOnly: true,
secure: [Link].NODE_ENV === 'production',
sameSite: 'strict',
maxAge: 7 * 24 * 60 * 60 * 1000
});

[Link](201).json({ success: true, accessToken,

user: { id: user._id, name: [Link], email: [Link] }
});
 } catch (err) { next(err); }
};

[Link] = async (req, res, next) => {

try {
const { email, password } = [Link];
const user = await [Link]({ email }).select('+password');

// Same error message whether email OR password is wrong.

// Different messages would let attackers enumerate valid emails.
if (!user || !(await [Link](password))) {
const err = new Error('Invalid email or password'); [Link] = 401;
return next(err);
}

const { accessToken, refreshToken } = generateTokens(user._id);

[Link]('refreshToken', refreshToken, {
httpOnly: true, secure: [Link].NODE_ENV === 'production',
sameSite: 'strict', maxAge: 7 * 24 * 60 * 60 * 1000
});
[Link]({ success: true, accessToken });
} catch (err) { next(err); }
};

6.2 Security Middleware

const helmet = require('helmet');
const rateLimit = require('express-rate-limit');
const mongoSanitize = require('express-mongo-sanitize');

// Helmet sets ~15 security-related HTTP response headers automatically

[Link](helmet());

// Rate limiting — prevent brute force and DDoS

const apiLimiter = rateLimit({
windowMs: 15 * 60 * 1000, // 15 minutes
max: 100,
message: { error: 'Too many requests. Try again in 15 minutes.' }
});
[Link]('/api/', apiLimiter);

// Tighter limit for auth endpoints (login attempts)

const authLimiter = rateLimit({ windowMs: 15 * 60 * 1000, max: 10 });
[Link]('/api/auth/', authLimiter);

// MongoDB injection sanitisation

// Strips $ and . from [Link], [Link], [Link].
// Without this: { email: { "$gt": "" } } matches ALL users — bypasses login!
[Link](mongoSanitize());
PART 7

Production Patterns

7.1 Environment Configuration

Never hardcode secrets, URLs, or environment-specific values in your code. Use environment
variables for everything that changes between environments (development, staging,
production). Validate required variables at startup — fail loudly immediately rather than
cryptically later.
// .env — NEVER commit this file (add to .gitignore)
// NODE_ENV=development
// PORT=5000
// MONGODB_URI=mongodb+srv://user:pass@[Link]/mydb
// JWT_SECRET=[Link](64).toString('hex')
// JWT_REFRESH_SECRET=another-64-byte-random-hex-string
// CLIENT_URL=[Link]

// .[Link] — ALWAYS commit this (without real values)

// NODE_ENV=
// PORT=
// MONGODB_URI=
// JWT_SECRET=

// config/[Link] — validate at startup

const required = ['MONGODB_URI', 'JWT_SECRET', 'JWT_REFRESH_SECRET'];
for (const key of required) {
if (![Link][key]) {
[Link](`FATAL: Missing required environment variable: ${key}`);
[Link](1);
}
}

7.2 Structured Logging with Pino

[Link] is not suitable for production. You need log levels (debug, info, warn, error),
structured JSON output so log aggregators like Datadog and Loki can parse and query your
logs, and request IDs to trace a single request through multiple log lines.
const pino = require('pino');
const pinoHttp = require('pino-http');

const logger = pino({

level: [Link].NODE_ENV === 'production' ? 'info' : 'debug',
transport: [Link].NODE_ENV !== 'production'
? { target: 'pino-pretty' } // human-readable colours in dev
: undefined // raw JSON in production (for aggregators)
});
 [Link](pinoHttp({ logger })); // logs every HTTP request automatically

// Structured logging throughout the app

[Link]({ userId: user._id, action: 'login' }, 'User logged in');
[Link]({ ip: [Link], attempts: 5 }, 'Multiple failed login
attempts');
[Link]({ err, requestId: [Link] }, 'Database query failed');

7.3 Graceful Shutdown

Graceful shutdown means stopping the acceptance of new connections, finishing all in-flight
requests, closing database connections, and then exiting. An abrupt shutdown (SIGKILL) risks
incomplete database writes and data corruption.
const server = [Link](PORT);

async function shutdown(signal) {

[Link](`${signal} received. Shutting down gracefully...`);

[Link](async () => {
try {
await [Link]();
[Link]('All connections closed. Exiting.');
[Link](0);
} catch (err) {
[Link]('Error during shutdown:', err);
[Link](1);
}
});

// Force exit after 10 seconds if graceful shutdown stalls

setTimeout(() => {
[Link]('Forced shutdown after timeout');
[Link](1);
}, 10000);
}

[Link]('SIGTERM', () => shutdown('SIGTERM')); // Docker/Kubernetes

[Link]('SIGINT', () => shutdown('SIGINT')); // Ctrl+C

7.4 Production Folder Structure

my-api/
├── src/
│ ├── config/
│ │ ├── [Link] # MongoDB connection
│ │ └── [Link] # Environment variable validation
│ ├── controllers/ # HTTP layer — read req, call service, write res
│ │ ├── [Link]
│ │ └── [Link]
 │ ├── services/ # Business logic — no req/res here
│ │ ├── [Link]
│ │ └── [Link]
│ ├── models/ # Mongoose schemas and models
│ │ └── [Link]
│ ├── routes/ # Express Router definitions
│ │ ├── [Link]
│ │ └── [Link]
│ ├── middleware/ # Custom middleware functions
│ │ ├── [Link]
│ │ └── [Link]
│ ├── validators/ # Zod schemas for input validation
│ │ └── [Link]
│ ├── utils/ # Pure utility functions (no side effects)
│ │ ├── [Link]
│ │ └── [Link]
│ └── [Link] # Entry point: connect DB, start server
├── tests/
│ ├── unit/
│ └── integration/
├── .env # SECRET — never commit
├── .[Link] # Template — always commit
├── .gitignore
├── [Link]
└── [Link]

7.5 HTTP Status Code Reference

2xx — Success
200 OK — standard success (GET, PUT)
201 Created — resource created successfully (POST)
204 No Content — success with no response body (DELETE)

4xx — Client Error (they did something wrong)

400 Bad Request — malformed JSON, invalid input
401 Unauthorized — not authenticated (missing or invalid token)
403 Forbidden — authenticated but not authorised (wrong role)
404 Not Found — resource does not exist
409 Conflict — duplicate resource (email already registered)
422 Unprocessable — valid format but fails business logic validation
429 Too Many Req. — rate limit exceeded

5xx — Server Error (you did something wrong)

500 Internal Error — unexpected crash
502 Bad Gateway — upstream service failed
503 Unavailable — server down or overloaded

This guide covers the complete path from JavaScript foundations through
production-grade [Link] backend engineering. Read each section with your
 terminal open — run every code example, break things intentionally, and
understand why they break. The understanding you build that way is permanent.
Shubhangam Singh · VIT Vellore · 23BKT0059