Zyro Recon System — SDE1

Interview Guide
(Top MNCs: Amazon, Google, Microsoft, Flipkart, PhonePe, Razorpay,
Paytm, Zepto, Swiggy)

How to use this guide: Each section has a "SAY THIS" block — that is exactly what you say in the interview. Read it out loud 5
times. Don't memorize word-for-word, understand the idea and say it naturally.

PART 1 — HOW TO INTRODUCE YOUR

PROJECT
30-Second Intro (When they say "Tell me about your projects")

SAY THIS: "I worked on Zyro, which is a financial transaction reconciliation engine. The core problem it solves is this — in UPI
payments, three systems record every transaction: the bank, NPCI, and our internal system. Zyro automatically ingests transaction
files from all three, normalizes them to a common format, does hash-based matching to detect discrepancies, and generates reports.
It's a backend Go service with PostgreSQL, Kafka, and AWS S3, processing millions of transactions in real time."

2-Minute Deep Intro (When they say "Walk me through the project")
SAY THIS: "So the project is called Zyro — it's a reconciliation management system for UPI transactions.

The problem we're solving is: every UPI payment gets recorded by three different systems — the bank debiting the user, NPCI which
is the network in the middle, and our internal payment system. At end of day, all three need to agree on every transaction. If they
don't, money can be lost or double-counted, and settlement between banks fails.

Zyro does this automatically. Here's the high-level flow:

First, banks and NPCI send CSV or Excel files to our SFTP server. We have a polling worker that picks these files up every 60
seconds. The file is then streamed through a template-based parser — we don't load the whole file into memory, we process 5000
rows at a time. Each row is converted to a canonical transaction format with a SHA256 hash.

The hash is the key idea. We compute SHA256 of the transaction ID and amount. If the bank and our internal system have the same
transaction ID with the same amount, their hashes will be identical — that's a MATCH. If amounts differ even by one paise, hashes
will be different — that's a MISMATCH.

A reconciliation worker runs every 5 seconds, does a merge join on sorted transaction lists, and updates the match status using a
bitfield. For mismatches, a rule engine evaluates whether the difference is within acceptable tolerance — like 2% for rounding
differences.

The whole thing is driven by configuration. Templates define file formats, rules define tolerance, and master config drives which
workers to spin up — so adding a new payment source requires zero code changes.

Tech stack is Go for concurrency, PostgreSQL with two separate DB connections, Kafka for event streaming with TLS, AWS S3 for
file storage, and SFTP for file ingestion."

PART 2 — THE FLOW (Easy to Explain Version)

The Simplest Way to Explain the Flow
Draw this mentally or on paper when explaining:
BANK/NPCI sends file
↓
SFTP Worker picks it up (every 60s)
↓
Parse file → 5000 rows at a time
↓
Convert to CanonicalTxn + compute SHA256 hash
↓
Store in DB (canonical_txn table)
↓
Create a "to-do" entry in recon_master
↓
Reconciliation Worker runs (every 5s)
→ Merge Join: compare INTERNAL vs BANK vs NPCI by recon_key
→ Same hash = MATCH, Different hash = MISMATCH
↓
Rule Engine checks mismatches
→ Within tolerance = OK, Outside = real discrepancy
↓
Generate report

Step-by-Step Flow Explanation (What to say for each step)

Step 1 — File Arrives

"Banks and NPCI deliver transaction files to our SFTP server. These are CSV or Excel files with columns like transaction ID, amount,
timestamp, and status. The SFTP worker polls every 60 seconds. It skips files modified less than a minute ago — that's a safety
check so we don't process a file that's still uploading."

Step 2 — Raw File Record Created

"Before we even start reading the file, we create a metadata record in the raw_file table. It stores the file name, which system sent
it (BANK/NPCI/INTERNAL), its status starting as INPROGRESS, and importantly — rows_inserted which starts at 0. This is our
checkpoint for resumability."

Step 3 — Template Resolution

"Each file has a different format. A bank file might have columns in a different order than an NPCI file. We resolve a template from
cache — the template tells us 'column 0 is transaction_id, column 1 is amount in paise, column 2 is timestamp in this format.'
Template lookup is from in-memory cache, no DB query needed."

Step 4 — Streaming Parse

"We stream the file in chunks of 5000 rows. Never load the whole file — a 10-million row file would be 200MB+ in memory. Each
chunk is parsed into ParsedRow structs using the template. Values are typed, validated, and transformed — for example, amounts
get multiplied by 100 if the source sends in rupees instead of paise."

Step 5 — Canonicalization + Hash

"Each ParsedRow becomes a CanonicalTxn. This is the normalize step — every system's data ends up in the same struct with the
same field names. The most important thing here is the hash: SHA256 of recon_key + amount. This is our transaction fingerprint.
Two records with the same recon_key and same amount will always produce the same hash."

Step 6 — Deduplication

"Before inserting, we check if this recon_key already exists for this system. If a bank resends a corrected file, we mark the old record
is_latest = false and insert the new one as is_latest = true. Reconciliation only looks at is_latest = true records."

Step 7 — ReconMaster Entry

"After insertion, we upsert a record in recon_master. This is our reconciliation state table. It has a system_mask bitfield showing
which systems have reported this transaction, and a pair_status_mask bitfield showing the comparison result between each pair.
Initially, status is PENDING."

Step 8 — Internal Sync (Parallel)

"Separately, an Internal Sync Worker pulls transactions from our own internal system via a read-replica every 10 seconds. It tracks its
last-pulled timestamp in internal_pull_state and only fetches new records since then — like a cursor. These also go through
canonicalization and get inserted with SystemID = INTERNAL."

Step 9 — Reconciliation Worker Runs

"Every 5 seconds, the Reconciliation Worker fetches all PENDING records from recon_master. It gets the INTERNAL transactions
and the BANK/NPCI transactions for those keys, sorts both lists by recon_key, and runs a merge join — same algorithm as merge
sort's merge step. For each matching key pair it compares hashes. Same hash = MATCH. Different hash = MISMATCH with a
HashMismatch object stored."

Step 10 — Pair Status Mask Update

 "The result is encoded into the pair_status_mask bitfield. For THREE_WAY reconciliation we have two pairs: INTERNAL vs
BANK (2 bits at offset 0) and INTERNAL vs NPCI (2 bits at offset 2). Values are 00=OPEN, 01=MATCH, 10=MISMATCH,
11=PENDING. Final status: all pairs MATCH → status=MATCH. Any MISMATCH → status=NO_MATCH."

Step 11 — Rule Engine

"Mismatches go to the rule engine. Rules are configured in DB and cached. A rule has conditions like: amount difference within 2%,
timestamp within 60 seconds, description fuzzy match. If all conditions pass, the mismatch is actually acceptable — maybe it's a
known rounding issue. The engine records its decision in rule_evaluations for audit."

Step 12 — Retry Handling

"If processing fails at row 4500 of 10000, the DB transaction rolls back for that batch and rows_inserted stays at 4500. The
RetryWorker picks up the failed file, opens the file stream, skips the first 4500 rows, and resumes from row 4501. No manual
intervention needed."

PART 3 — TECH STACK (How to explain each

choice)
Technology One-line answer for "why did you use this?"
"Concurrency-first language — goroutines are perfect for multiple polling workers. Single
Go
binary deployment, no JVM overhead."
"ACID transactions critical for financial data. Bitfield operations native. Two separate pools:
PostgreSQL
write DB + read-only external replica."
"Decouples file events from processing. TLS for data-in-transit security. DLQ ensures no
Kafka
events are silently lost."
"Production file storage — durable and accessible across instances. Local filesystem for
AWS S3
dev. Abstracted behind interface so switching needs only config change."
"Industry standard for B2B financial file exchange. Banks don't send APIs — they send
SFTP
files via SFTP."
In-memory "Templates and rules don't change per request. Without cache, 1M transactions = 1M DB
Cache queries just for template lookup. 2-hour TTL with atomic swap — lock-free reads."
"Banks send both CSV and Excel formats. excelize is the most mature Go library with
excelize
streaming Excel support."
"20-30% faster than standard database/sql. Native PostgreSQL protocol, better pool
pgx/v5
management."
"YAML config with env variable override — same binary runs in dev/staging/prod just by
Viper
swapping config."

PART 4 — SDE1 TECHNICAL INTERVIEW

QUESTIONS
 SECTION A — Project Understanding (Asked in every company)
Q1: Tell me about your most complex project.

SAY THIS: "The most complex part of Zyro for me was the reconciliation matching logic combined with the bitfield state management.
The challenge was: we have three systems sending data asynchronously, and we need to track the exact reconciliation status of
every transaction across every pair of systems — efficiently, in a single DB column.

The solution was a two-level bitfield. system_mask tracks which systems have arrived, and pair_status_mask tracks the 2-bit
match result for each system pair at specific bit offsets. The reconciliation worker does a merge join on sorted data — O(n log n)
instead of O(n²) nested loops.

What made it complex was the asynchronous nature — BANK data might arrive before INTERNAL data, so we needed the PENDING
state and a retry loop, rather than waiting or blocking."

Q2: What was the biggest challenge you faced?

SAY THIS: "The biggest challenge was designing for failures. Financial processing cannot lose data. Three specific problems we
solved:

First — partial file processing. If a server crashes at row 4999 of 10000, you can't re-process from the beginning or you'd double-
count 4999 transactions. Our solution: rows_inserted checkpoint + resume from offset.

Second — concurrent file processing. If two service instances try to process the same file, you get duplicates. Our solution: database
lease (lease_expires_at) that each worker sets and extends every batch.

Third — stale data. Banks resend corrected files. is_latest flag ensures only the newest version is used for reconciliation while old
versions remain for audit."

Q3: What would you do differently if you built this again?

SAY THIS: "A few things. First, I'd add distributed tracing from the start — right now debugging a failed reconciliation means manually
joining multiple tables. OpenTelemetry traces would connect the file ingestion, canonicalization, and reconciliation steps automatically.

Second, the rule engine conditions are stored as JSON in the DB — parsing and evaluating JSON on every mismatch has overhead.
I'd consider compiling rules into an AST or using a proper expression evaluator.

Third, the polling workers use fixed intervals (5s, 10s, 60s). I'd replace these with event-driven triggers via Kafka — process
immediately when a file arrives instead of waiting up to 5 seconds."

Q4: How does the system scale?

SAY THIS: "Horizontally — multiple instances of the service can run simultaneously. The lease system on raw_file ensures only
one instance processes any given file. The reconciliation worker uses DB-level locking to claim batches of recon_master records, so
multiple workers can reconcile different transactions in parallel.

The cache layer means adding more instances doesn't increase DB load from config lookups. The streaming file processing means
memory usage stays constant regardless of file size. The streaming pipeline processes each chunk, commits it, and moves on —
memory from the previous chunk is freed."

SECTION B — Data Structures & Algorithms (Google, Microsoft

focus)
Q5: Explain the merge join algorithm you used. What is its time complexity?

SAY THIS: "MergeJoin is the same as the merge step in merge sort. You have two sorted arrays. You maintain two pointers — one
for each array. At each step, you compare the current elements. If they match, process the pair and advance both pointers. If one is
smaller, it means that key only exists in one list, advance that pointer. This is O(n) for the join step itself. Total complexity is O(n log n)
for sorting both arrays first, then O(n) for the join. Compared to a naive nested loop which would be O(n²), for 1 million transactions
this is the difference between 1 million operations and 1 trillion operations."

Q6: Why use a bitfield for system_mask and pair_status_mask instead of separate columns?

SAY THIS: "Three reasons. First, efficiency — one integer column stores the presence/status of 64 different systems. Separate
columns would require a schema change every time a new system is added. Second, atomic updates — setting multiple pair statuses
in one integer update is a single atomic operation. With separate columns, updating 3 columns is 3 operations that need a
transaction. Third, query speed — checking 'has system BANK arrived?' is system_mask & BANK_BIT != 0, a single CPU
instruction. Querying a boolean column would require an index lookup."

Q7: How does hashing help in reconciliation? Why SHA256 specifically?

SAY THIS: "The hash acts as a transaction fingerprint. Instead of comparing each field individually — amount, timestamp, status,
reference numbers — we compare two strings. Same hash means all fields agree. Different hash means something differs, and we
hand it to the rule engine.

SHA256 specifically because: it's deterministic (same input always produces same output), collision-resistant (two different inputs
producing the same hash is computationally infeasible), and fixed-length (always 64 hex characters regardless of input size). For
financial data, SHA256's collision resistance is important — a collision would mean two different transactions appear to match."

Q8: How does your streaming pipeline work? Explain the producer-consumer pattern.
SAY THIS: "We use Go's [Link] which creates a synchronous in-memory pipe — a writer and a reader connected. The producer
(SFTP file reader) runs in a goroutine and writes chunks to the pipe's write end. The consumer (parser) reads from the pipe's read
end in the main goroutine. They run concurrently and synchronize automatically — the producer blocks when the consumer hasn't
caught up, the consumer blocks when the producer hasn't sent data yet. Error propagation works through Go's channel pattern — if
the SFTP connection drops, the producer goroutine returns an error, the pipe closes, and the consumer gets an error on next read.
No partial data leaks."

Q9: What is the time and space complexity of processing a file with N rows?

SAY THIS: "Time: O(N) — each row is processed once through parsing, canonicalization, and insertion. The merge join is O(N log N)
per reconciliation batch but that's a separate step.

Space: O(1) relative to N — we process in chunks of 5000 rows. Memory usage is constant at any file size. The 5000-row chunk is
processed, committed, garbage collected, then the next chunk is loaded. If the file has 10 million rows, memory usage is the same as
if it had 10000 rows."

Q10: Explain how the deduplication logic works. What data structure would you use to find duplicates efficiently?

SAY THIS: "We query the DB for all existing recon_keys from the incoming batch that already exist in canonical_txn for that
system. The result is a set of duplicate keys. Then for the incoming batch, we split it into: keys already in DB (duplicates) and new
keys. For duplicates, we run UPDATE SET is_latest=false, then INSERT the new version. For new keys, just INSERT.

In code, we use a Go map (map[string]bool) to store the duplicate set — O(1) lookup. Without the map, we'd need O(n²) nested
loops or repeated DB queries. The initial DB query for duplicates is a single WHERE recon_key IN (...) query with the full
batch."

SECTION C — System Design Basics (Amazon, Flipkart focus)

Q11: How would you design this system from scratch? (High-level)
SAY THIS: "I'd identify the core requirements first. Inputs: transaction files from multiple systems. Output: reconciliation status per
transaction. Constraints: no data loss, resumable processing, handle millions of transactions.

Key decisions:

Streaming over batch loading — files can be huge

Hash-based matching over field-by-field — O(1) comparison per transaction pair
Bitfield for state — compact, atomic, extensible
Separate DB pools — read-replica isolation
Polling workers over synchronous API — payment networks push files, not events
Config-driven workers — adding sources without code changes

The main trade-off is consistency vs. availability. We chose eventual consistency — PENDING state with polling retries — over
blocking until all three systems have data, which would create deadlocks when any system is delayed."

Q12: How do you handle the scenario where BANK data arrives but INTERNAL data hasn't synced yet?

SAY THIS: "This is exactly why we have the PENDING state. When BANK file is ingested, we create a recon_master entry with
system_mask = BANK_BIT and status = PENDING. The reconciliation worker sees this is PENDING, checks if INTERNAL data
exists for those keys, and if not, skips and tries again in 5 seconds. Meanwhile, the InternalSyncWorker is independently polling the
internal read-replica every 10 seconds and inserting INTERNAL canonical records. Once both sides are present, the next
reconciliation poll finds the pair and performs the match. No blocking, no waiting — pure eventual consistency."

Q13: What happens if the same file is accidentally ingested twice?

SAY THIS: "Two protection layers. First, at the raw_file level — we could add a unique constraint on file_name + system_id +
file_date to reject the second insertion at DB level. Second, even if two raw_file records exist, the deduplication in ProcessorService
handles it — when the second file's rows are being inserted, each recon_key already exists, so the old record is marked
is_latest=false and the new identical data is inserted as is_latest=true. Net result: same data, same hash, reconciliation produces
same result. The second ingestion is idempotent."

Q14: How would you add a new payment system (say, Visa) without code changes?
SAY THIS: "That's a design goal we achieved. You'd need to:

1. Insert a row in system_registry for Visa with a new bit index

2. Insert rows in recon_pair_ref for the new pairs (e.g., INTERNAL vs VISA)
3. Insert a template in file_template and file_template_column for Visa's file format
4. Insert a row in source_config for the Visa SFTP details
5. If Visa data needs internal sync, insert a row in master_config

Then restart the service. The cache reloads, the SFTP worker discovers the new source, the template registry resolves the new file
format, and the reconciliation worker automatically handles the new pair because pairs come from the DB. Zero code changes."

Q15: How does your caching strategy work? What happens when a cache entry is stale?

SAY THIS: "Each cache runs a background goroutine that reloads from DB every 2 hours. The loaded data is stored in an
[Link] — this is Go's lock-free atomic reference. The swap is atomic: one instruction replaces the old cache snapshot with
the new one. Any goroutine reading from cache either reads the old snapshot or the new one — never a half-updated inconsistent
state.

For stale data: if a new template is added in DB and the cache hasn't refreshed yet (within the 2-hour window), that template won't be
visible. This is an accepted trade-off for our use case — templates don't change mid-day. For rule changes that need immediate
effect, you'd need to add a manual cache invalidation endpoint."

SECTION D — Databases & Transactions (All MNCs)

Q16: Why do you wrap file processing in a database transaction?

SAY THIS: "Atomicity. If we're inserting a batch of 5000 canonical transactions and the server crashes at transaction 4999, without a
DB transaction those 4999 rows are in the DB permanently. But rows_inserted would also have been updated... or maybe not,
depending on when the crash happened. This creates an inconsistent state that's very hard to recover from.

With a DB transaction: the batch insert, the rows_inserted update, and any recon_master updates all happen atomically. Either all
5000 succeed and rows_inserted jumps from 0 to 5000, or ALL roll back and rows_inserted stays at 0. The retry can safely restart
from 0."

Q17: Why two separate database connections — internal and external?

SAY THIS: "The external DB is the bank/switch system's own database. We have read-only access to it via a read replica. Keeping it
as a separate connection pool has several benefits: it enforces that we only read from it (we have no write credentials), connection
pool exhaustion on one doesn't affect the other, and we can configure different pool sizes — internal might need 20 connections for
writes, external read-replica might need 5 since it's only used by InternalSyncWorker."
 Q18: What is the N+1 query problem and does your system have it?

SAY THIS: "N+1 problem is when you run 1 query to get N records and then N separate queries to get related data for each record —
N+1 queries total instead of 1 join query. We avoided it by bulk operations. For example, the reconciliation worker doesn't fetch one
transaction at a time — it collects all pending recon_keys, then does a single WHERE recon_key IN (...) query to fetch all
transactions at once. Similarly, batch inserts use PostgreSQL's bulk insert with unnest — one query for 5000 rows instead of 5000
single-row inserts."

Q19: What is an index and would you add any to this system?

SAY THIS: "An index is a sorted data structure (usually B-tree) on a column that speeds up lookups. Without it, the DB scans every
row — O(n). With it — O(log n).

For this system I'd add:

canonical_txn(recon_key, system_id) — most queries are by recon_key + system

canonical_txn(recon_key, is_latest) — reconciliation only reads is_latest=true
recon_master(status) — workers poll for status=PENDING frequently
raw_file(status, retry_count) — retry worker queries by these fields
raw_file(lease_expires_at) where lease has expired — for lease expiry cleanup"

SECTION E — Concurrency & Go-Specific (Google, Microsoft focus)

Q20: How do multiple workers avoid processing the same file?

SAY THIS: "Database-level lease. When a worker picks up a file, it does a SELECT...FOR UPDATE on the raw_file row, sets
processing_by = worker_id and lease_expires_at = now + 10 minutes. Other workers filter out files where
lease_expires_at > now. Every batch the active worker extends the lease. If the worker dies, the lease expires after 10 minutes
and another worker can pick it up. This is a distributed lock implemented in PostgreSQL — no need for Redis or Zookeeper."

Q21: How do you prevent race conditions in your Go code?

SAY THIS: "Primarily through design. The cache uses [Link] for lock-free reads — no mutex needed. The DB is the source
of truth for coordination between workers — PostgreSQL's ACID guarantees handle the critical sections. Within a single worker,
processing is sequential within a file — no shared mutable state between goroutines. Where goroutines need to communicate, we
use channels. The streaming pipeline producer-consumer communicates through [Link] which handles synchronization internally."

Q22: What is a goroutine and how is it different from a thread?

 SAY THIS: "A goroutine is a lightweight concurrent unit of execution managed by Go's runtime, not the OS. A traditional OS thread
typically uses 1-2 MB of stack. A goroutine starts with 8KB of stack and grows dynamically. Go can run thousands of goroutines on a
handful of OS threads using M:N scheduling — M goroutines mapped to N OS threads. In our system, we have goroutines for each
cache refresh, each polling worker, the streaming pipeline producer — maybe 10-20 goroutines total. If these were OS threads, each
polling interval would create/destroy threads with significant overhead. Goroutines make this essentially free."

Q23: Explain how Go channels are used in your system.

SAY THIS: "The most important use is the Ready() channel on each cache. Each cache's Ready() returns a channel. When the
cache finishes its first data load, it closes the channel. [Link] does <-[Link]() which blocks until the channel closes. This
is Go's idiomatic way to signal 'I'm done' — a closed channel is always readable and returns immediately. We use this for all four
caches before starting workers, ensuring no worker starts with stale/empty cache."

SECTION F — Situation-Based Questions (Amazon LP style, Flipkart,

PhonePe)
Q24: Tell me about a time you had to make a trade-off between performance and correctness.

SAY THIS (STAR format): "Situation: The reconciliation worker was initially doing field-by-field comparison for every transaction pair
— comparing amount, timestamp, status, reference numbers separately. For 100,000 mismatched pairs it was slow.

Task: We needed to speed up the detection step while keeping the detailed analysis for genuine mismatches.

Action: I proposed using SHA256 hashing. Compute a hash of the critical fields once per transaction at ingestion time. Reconciliation
just compares two strings — O(1) per pair. The detailed rule engine evaluation only runs on the subset that actually mismatches,
which is typically less than 1% of transactions.

Result: Reconciliation time dropped significantly. The trade-off was we can't tell WHY records differ from the hash alone — but that's
exactly what the rule engine is for. We got both: fast detection and detailed analysis, on different subsets of data."

Q25: Tell me about a time you had to ensure data was never lost.
SAY THIS (STAR format): "Situation: We had a scenario where file processing could fail midway — network drops, server restarts,
OOM kills. If we lost progress, we'd either miss transactions or double-count them.

Task: Design a failure recovery mechanism with exactly-once semantics.

Action: I designed the rows_inserted checkpoint system. Every batch of 5000 rows is wrapped in a DB transaction.
rows_inserted only advances on successful commit. If the server crashes, on restart the RetryWorker reads rows_inserted,
opens the file, skips exactly that many rows using the file reader's offset, and resumes. The file reader supports seeking to a byte
offset, so we skip already-processed rows without re-reading them.

Result: Any failure is recoverable. We've processed files of 2 million rows that failed at row 1.5 million and resumed perfectly."

Q26: Describe a situation where you improved system performance.

SAY THIS (STAR format): "Situation: The initial implementation of canonical transaction insertion was doing one INSERT per row
— for a 100,000 row file that's 100,000 round trips to the database.

Task: Reduce database load without changing the correctness guarantees.

Action: I switched to bulk inserts using PostgreSQL's unnest pattern — pass arrays of values and unnest them into rows in a single
query. One query for 5000 rows instead of 5000 queries. We also moved the deduplication check to a single WHERE recon_key =
ANY($1) query for the whole batch instead of per-row lookups.

Result: File ingestion time for a 100,000 row file dropped from minutes to seconds. DB connection usage dropped dramatically
because each batch uses the connection for seconds instead of minutes."

Q27: How did you handle a bug that only appeared in production?

SAY THIS (STAR format): "Situation: In production, some transactions were showing as MISMATCH even though the bank and our
internal system had the same amount. This wasn't reproducible locally.

Task: Diagnose and fix without being able to reproduce locally.

Action: I added detailed logging to the hash computation, logging the exact input string to SHA256. I discovered that our internal
system was sending amounts in rupees (₹ 500) while the bank was sending in paise (50000). Locally our test data was all in paise so
we never caught it. The hash inputs were '500' vs '50000' — completely different.

Result: Fixed by adding a MULTIPLY_100 transform rule to the internal system's template. Also added a validation step that checks
all amounts are in paise range (> 100) before hashing to catch this class of bug earlier."

Q28: Tell me about a time you had to work with an ambiguous requirement.
SAY THIS (STAR format): "Situation: The requirement said 'reconcile transactions across systems' but didn't specify what to do
when a BANK transaction exists but INTERNAL doesn't have it yet.

Task: Define the correct behavior and implement it.

Action: I analyzed the possible states: INTERNAL could arrive first, BANK could arrive first, or they could arrive simultaneously. I
proposed three states — PENDING (waiting for data), MATCH (all systems agree), NO_MATCH (definitive mismatch). The PENDING
state allows asynchronous arrival without blocking. I documented this as the 'eventual consistency model' for the team.

Result: The PENDING + polling design handled all timing scenarios. It also naturally handles the retry case — if INTERNAL data is
late by hours, the reconciliation worker just keeps polling until it arrives."

Q29: Tell me about a time you had to explain a technical decision to a non-technical stakeholder.

SAY THIS (STAR format): "Situation: A product manager asked why we couldn't show reconciliation results instantly after a file is
uploaded.

Task: Explain eventual consistency vs. immediate consistency in a way that made sense.

Action: I used an analogy — it's like comparing two bank statements. You can't compare them until both arrive. Our internal
statement (InternalSyncWorker) takes up to 10 seconds to arrive after a BANK file is uploaded. The reconciliation itself takes up to 5
more seconds. So the minimum latency is 15 seconds. I showed a simple timeline diagram showing when each piece of data arrives.

Result: The PM understood and we agreed on a 'status: processing' state in the UI with a webhook notification when reconciliation
completes. We also added a max-latency SLA of 30 seconds for 99th percentile."

Q30: Describe a situation where you had to balance adding new features vs. fixing technical debt.

SAY THIS (STAR format): "Situation: We needed to add support for a new file format (pipe-delimited) while also having a known
issue — the template column mapping used 0-based index which caused off-by-one errors when files had a header row.

Task: Decide whether to fix the debt first or add the feature.

Action: I fixed the header handling bug first. The reason: if we added a new file format with the same bug, we'd have two buggy
implementations to fix later. The fix was small — add a SkipRows(1) call when [Link] = true. Once the parser
was correct, adding pipe-delimited support was trivial — just a new case in the reader factory.

Result: Fixed both in one PR. The key insight was that the debt was in the foundation — the parser — not in a peripheral feature.
Fixing the foundation first made the feature cleaner."

SECTION G — Fintech-Specific Questions (PhonePe, Paytm,

Razorpay, BankBazaar)
Q31: What is UPI reconciliation and why does it matter?
SAY THIS: "UPI (Unified Payments Interface) is India's real-time payment network run by NPCI. Every UPI transaction passes
through multiple systems: the user's bank debits the amount, NPCI routes the payment, and the merchant's bank credits it. Each
system maintains its own record. Reconciliation verifies all three records agree.

It matters for three reasons: settlement — banks and NPCI settle net positions at end of day, and disputed transactions can't be
settled; fraud detection — if INTERNAL shows ₹ 500 but bank shows ₹ 5000, someone manipulated a transaction; and regulatory
compliance — RBI mandates that all payment systems maintain reconciliation records."

Q32: Why store amounts in paise instead of rupees?

SAY THIS: "Floating-point arithmetic is dangerous for money. Try 0.1 + 0.2 in Python — you get 0.30000000000000004. If we
stored ₹ 500.01 as a float and compared it to another float representation of ₹ 500.01, they might not be equal due to IEEE 754
precision. By storing in paise as integers (₹ 500.01 = 50001 paise), all arithmetic is exact integer math. This is standard practice in
fintech globally — Stripe, Razorpay, and most payment systems store amounts in the smallest currency unit."

Q33: What is a DLQ and why is it important in financial systems?

SAY THIS: "DLQ stands for Dead Letter Queue. In Kafka, if a consumer fails to process a message after configured retries, instead of
dropping it, the message is routed to a special DLQ topic. In financial systems, dropping a message could mean a transaction is
never reconciled — meaning money is unaccounted for. The DLQ gives us a safety net: a team member can inspect failed
messages, understand why they failed (maybe malformed data), fix the issue, and reprocess the DLQ messages. Silent drops in
financial systems are unacceptable."

Q34: What is the difference between reconciliation and settlement?

SAY THIS: "Reconciliation is the verification step — confirming that all parties agree on what transactions happened. Settlement is
the actual money movement that happens after verification.

Example: At end of business day, Bank A has 1000 transactions where users of Bank A paid merchants on Bank B via NPCI.
Reconciliation verifies all 1000 are correctly recorded across NPCI and both banks. Settlement then calculates the net amount Bank
A owes Bank B (or vice versa) and transfers it in bulk — instead of 1000 individual transfers.

Our system handles reconciliation. Settlement uses our reconciliation output to know which transactions are valid and their confirmed
amounts."

SECTION H — Code/Design Questions (Google, Microsoft focus)

Q35: How would you design the CanonicalTxn struct? What fields would you include?
SAY THIS: "Core fields: system_id (which system sent this), recon_key (common transaction ID), amount (in paise, as int64),
txn_time (timestamp), hash (SHA256 fingerprint), payload_json (original raw data for audit), is_latest (true for current
version), and timestamps.

Key design decisions: amount as int64 not float64 — financial amounts need exact arithmetic. payload_json as raw string not
a struct — the raw format varies by system and we don't want to lose original data. is_latest as boolean not a version number —
we only ever care about current vs. historical, not which version number something is."

Q36: How would you write a test for the reconciliation merge join logic?

SAY THIS: "I'd write table-driven tests in Go with these cases:

All match: same keys, same hashes → all MATCH

All mismatch: same keys, different hashes → all MISMATCH
Only internal: keys in INTERNAL only → ONLY_INTERNAL
Only external: keys in EXTERNAL only → ONLY_EXTERNAL
Mixed: some match, some mismatch, some one-sided → correct counts in each bucket
Empty left: external list has data, internal is empty → all ONLY_EXTERNAL
Empty right: internal has data, external is empty → all ONLY_INTERNAL
Empty both: no panic, returns empty results

I'd use testify/assert for clean error messages. No DB needed — the merge join function only takes sorted slices as input,
making it a pure unit test."

Q37: How would you add rate limiting to the SFTP polling?

SAY THIS: "Go has [Link] for fixed-interval polling — already used for the 60-second SFTP poll. For rate limiting the number
of concurrent file processings (e.g., max 5 files at once), I'd use a buffered channel as a semaphore:

sem := make(chan struct{}, 5) // max 5 concurrent

for _, file := range files {
sem <- struct{}{} // acquire slot
go func(f File) {
defer func() { <-sem }() // release slot
process(f)
}(file)
}

This is Go's idiomatic semaphore pattern — no external libraries needed."

SECTION I — Go Language Deep Dive (Google, Microsoft, Uber

focus)
 Q38: What is an interface in Go and how do you use it in this project?

SAY THIS: "An interface in Go defines a set of method signatures. Any type that implements all those methods automatically satisfies
the interface — no explicit declaration needed. This is called implicit interface satisfaction.

In this project we use interfaces everywhere. The Cache interface has Start(ctx) and Ready() methods — all four caches
implement it. The Storage interface has DownloadStream and UploadStream — both local filesystem and S3 implement it. The
Reader interface abstracts CSV and Excel readers. Because of interfaces, the ingestion service doesn't know whether it's reading
from S3 or a local disk — it just calls [Link](). Switching from local to S3 in production requires zero code
changes, just config."

Q39: What is defer in Go? Where is it used in this project?

SAY THIS: "defer schedules a function to run when the surrounding function returns — regardless of whether it returns normally or
via panic. It runs in LIFO (last in, first out) order if there are multiple defers.

In this project defer is critical for resource cleanup. When we open an SFTP file stream, we defer [Link]() immediately
after opening — this guarantees the file handle is released even if parsing fails halfway. For DB transactions, we defer
[Link]() right after starting the transaction — if the function returns without calling [Link](), the transaction rolls back
automatically. This prevents connection leaks that would starve the connection pool."

Q40: What is [Link] and why is it passed everywhere?

SAY THIS: "Context carries deadlines, cancellation signals, and request-scoped values across API boundaries. Every worker in this
project accepts a [Link] parameter.

Two main uses here: First, cancellation propagation — when the main process receives a SIGTERM (shutdown signal), it cancels the
root context. Every worker's select loop checks [Link]() and gracefully stops. Without context, workers would keep running after
the process is supposed to shut down. Second, timeout control — a database query that hangs forever can be cancelled by setting a
context deadline. For a financial system, a hanging DB query should timeout and retry, not block forever."

Q41: What is a goroutine leak and how would you detect it in this system?

SAY THIS: "A goroutine leak is when a goroutine is started but never terminates — it keeps consuming memory and CPU forever.
Common cause: a goroutine waiting on a channel that nobody will ever send to, or waiting on a network call with no timeout.

In this system a potential leak is in the streaming pipeline — the producer goroutine writes to the pipe, the consumer reads. If the
consumer panics and stops reading without closing the pipe, the producer goroutine blocks forever trying to write. We prevent this by
propagating errors through the context and ensuring both sides properly close the pipe on any error.

Detection: [Link]() tells how many goroutines are running. If this count grows over time without bound, you
have a leak. Tools like pprof can show which goroutines are stuck and why."
 Q42: What is select in Go? How is it used in the workers?

SAY THIS: "select is like a switch statement for channels — it waits on multiple channel operations and executes whichever one is
ready first. If multiple are ready simultaneously, it picks one randomly.

In the polling workers, the core pattern is:

for {
select {
case <-[Link]():
return // Shutdown signal received
case <-ticker.C:
doWork() // Poll interval fired
}
}

This lets the worker simultaneously wait for its next poll interval AND a shutdown signal. Without select, you'd have to choose
between checking cancellation and doing work — you can't wait for both at the same time."

Q43: What is the difference between a mutex and a channel? When would you use each?

SAY THIS: "A mutex ([Link]) protects shared state — you lock before reading/writing shared data, unlock after. A channel
transfers data between goroutines — one goroutine sends, another receives.

Rule of thumb: use a channel when you want to communicate data between goroutines. Use a mutex when multiple goroutines need
to read/write the same shared variable.

In this project: the cache uses [Link] (not even a mutex) for reads because reads vastly outnumber writes —
[Link] is faster. For the DB connection pool, pgx uses internal mutexes for pool management. The streaming pipeline uses
[Link] which is channel-like — no shared state, data flows one direction."

Q44: What is panic and recover in Go?

SAY THIS: "Panic is Go's mechanism for unrecoverable errors — it stops normal execution, runs deferred functions, and if not
recovered, crashes the program with a stack trace. recover() inside a deferred function can catch a panic and return execution to
normal.

In this project's workers, we'd wrap the main polling loop body in a recover to prevent one bad transaction from crashing the entire
service. For example, if a specific recon_key causes a nil pointer panic in the merge join, we recover, log the error with the recon_key
details, and continue processing the rest. In financial systems, crashing the whole reconciliation process because of one malformed
record is worse than skipping that record and alerting."

Q45: Explain Go's module system. What is [Link]?

SAY THIS: "[Link] defines the module name and lists all direct dependencies with their exact versions. [Link] stores the
cryptographic hash of every dependency — this ensures you always get exactly the same code regardless of when you build. This is
reproducible builds.

In this project, key dependencies in [Link]: jackc/pgx/v5 for PostgreSQL, segmentio/kafka-go for Kafka,
xuri/excelize/v2 for Excel parsing, spf13/viper for config, pkg/sftp for SFTP. When you run go mod tidy, it removes
unused imports and adds missing ones. When you go build, it downloads exactly the versions in [Link] — no surprises."

SECTION J — Kafka & Messaging Deep Dive (All companies with

distributed systems)
Q46: What is Kafka and why did you use it over a simple REST API call?

SAY THIS: "Kafka is a distributed event streaming platform. You publish events to topics and consumers subscribe to those topics.
The key difference from REST: REST is synchronous — the caller waits for a response. Kafka is asynchronous — the publisher
sends an event and continues; consumers process it independently.

For this system, when a file arrives on SFTP, we need to trigger ingestion. With REST, if the ingestion service is down, the event is
lost. With Kafka, the event sits in the topic until the consumer is ready — durability is built-in. Also, multiple consumers can read the
same event independently — if we want to both ingest and audit log a file arrival, both consumers get the same event without
coupling."

Q47: What is a Kafka consumer group?

SAY THIS: "A consumer group is a group of consumers that share the work of consuming a topic. Kafka partitions the topic, and each
partition is assigned to exactly one consumer in the group at a time. This means: if you have 10 partitions and 5 consumers in a
group, each consumer handles 2 partitions — parallel processing. If a consumer dies, its partitions are rebalanced to surviving
consumers automatically.

In this project, all instances of the Zyro service belong to the same consumer group ID (recon-consumer-group). This ensures
each file upload event is processed by exactly one service instance — no duplicates."

Q48: What is at-least-once delivery in Kafka? How does your system handle duplicates?

SAY THIS: "Kafka guarantees at-least-once delivery by default — a message might be delivered more than once if a consumer
processes a message but crashes before committing the offset. On restart, Kafka re-delivers from the last committed offset.

Our system handles this with idempotency. If a file is re-processed, the deduplication logic (is_latest flag and recon_key
uniqueness check) ensures we don't double-count transactions. The reconciliation result is the same whether a file is processed once
or ten times. This is the correct approach — we accept at-least-once delivery from Kafka but make our processing idempotent."
 Q49: What is a Kafka offset?

SAY THIS: "An offset is a unique sequential number assigned to each message in a Kafka partition — like a line number in a file.
Consumers track which offset they've consumed. When a consumer restarts, it resumes from its last committed offset.

This is similar to our rows_inserted checkpoint in file processing — both are cursor-based resumption patterns. The difference:
Kafka manages offsets for you, while our rows_inserted is a manual cursor we maintain in PostgreSQL."

Q50: What does TLS do in your Kafka configuration?

SAY THIS: "TLS (Transport Layer Security) encrypts data in transit between Kafka clients and brokers. Without TLS, transaction data
flowing through Kafka would be in plaintext — anyone with network access between our service and the Kafka broker could read
sensitive financial data.

Our configuration uses mutual TLS — both client and server authenticate each other. The client presents a certificate ([Link]),
the server presents its certificate verified against our CA ([Link]). This prevents man-in-the-middle attacks. For financial data, RBI
guidelines require encryption in transit, so TLS isn't optional."

SECTION K — Design Patterns & Low-Level Design (Google,

Microsoft focus)
Q51: What design patterns did you use in this project?

SAY THIS: "Several patterns:

Factory Pattern — worker/factory_worker.go creates InternalSyncWorker instances based on master_config records from the
DB. The caller doesn't know which concrete worker is created, just that it gets a Worker interface. Same with storage/[Link]
— creates either LocalStorage or S3Storage based on config.

Strategy Pattern — The rule engine uses different evaluation strategies (EQUAL, AMOUNT_TOLERANCE, TIME_RANGE, FUZZY)
that can be swapped based on the condition type. Each condition type is a strategy for comparing two field values.

Template Method Pattern — The generic parser defines the skeleton (read → map → validate → transform → return ParsedRow)
but the concrete reader (CSV vs Excel) implements the actual reading step. The algorithm structure is fixed; only one step varies.

Module Pattern / DI Container — Every major subsystem has a [Link] that acts as a factory composing internal
dependencies. [Link] only talks to modules, never to internal service implementations directly."

Q52: What is Dependency Injection and why does it matter for testing?
SAY THIS: "Dependency Injection means a component receives its dependencies from outside rather than creating them internally.
Instead of [Link] = [Link](...) inside the service, you pass the db connection: NewService(db *[Link]).

Why it matters for testing: in tests you can pass a mock or stub instead of a real DB connection. For example, to test the
reconciliation merge join logic, you don't need a real database — you inject a fake repository that returns hardcoded transactions.
The merge join logic is tested in complete isolation. Without DI, your service would create a real DB connection in its constructor and
every test would need a real database."

Q53: Explain the Module pattern used in this project.

SAY THIS: "Each major subsystem has a [Link] file. The module is a struct that holds all the services and repositories for that
subsystem. It has a NewModule(dependencies...) constructor that creates and wires everything internally. The module exposes
only the public services that other modules need.

For example, [Link](ctx, repo) creates all four caches, wires them with the repository, and returns a CacheModule
struct. [Link] calls [Link]() and gets back the module. Then [Link](db, repo, ingestion,
cache) receives the cache module and uses it. [Link] never directly instantiates individual cache objects — it always goes
through the module.

This pattern makes [Link] the only place with wiring knowledge. Each module is independently testable and replaceable."

Q54: How does the Template Method pattern work in the file parser?

SAY THIS: "The generic parser defines the overall algorithm: loop through rows, for each row call the row mapper, the row mapper
applies column index lookup, validation, transformation, and field name mapping. This sequence is fixed.

What varies is the reading part — CSV files and Excel files have completely different binary formats. We have a Reader interface
with ReadChunk(n) method. CSVReader implements it using Go's encoding/csv. ExcelReader implements it using excelize. The
parser just calls [Link](5000) and gets back [][]string regardless of file format. The template method (the
parsing algorithm) is in the generic parser; the step that varies (how to read the file) is in the concrete reader."

Q55: What is the Observer / Pub-Sub pattern? Where is it in this project?

SAY THIS: "Observer pattern: when state changes, all registered observers are notified. Pub-Sub is similar — publishers emit events
to topics, subscribers receive them without knowing who published.

In this project, Kafka implements Pub-Sub. When a file arrives, an event is published to a Kafka topic. Any number of consumers
(subscribers) can react — the ingestion consumer processes the file, an audit consumer logs the event, a monitoring consumer
records metrics. Publishers and subscribers are completely decoupled.

The cache's Ready() channel is a simpler observer — [Link] 'subscribes' by waiting on the channel, the cache 'publishes' by
closing the channel when ready."
 Q56: How would you design a REST API for this system?

SAY THIS: "I'd expose these endpoints:

File operations:

POST /files/upload — upload a new file, returns file_id

GET /files/{file_id}/status — get processing status and rows_inserted
POST /files/{file_id}/retry — manually trigger retry for a failed file

Reconciliation:

GET /recon/{recon_key} — get reconciliation status for a transaction

GET /recon/summary?date=2024-06-01&system=BANK — summary by date/system
GET /recon/mismatches?date=2024-06-01 — all mismatches for a date

Reports:

POST /reports/generate — trigger report generation, returns job_id

GET /reports/{job_id} — get report status or download URL

Admin:

POST /cache/refresh — force cache reload

GET /health — health check for load balancer

Authentication would use JWT tokens for service-to-service and OAuth for user-facing endpoints."

SECTION L — Testing Questions (All companies)

Q57: How did you test this project? What types of tests did you write?

SAY THIS: "Three levels of testing:

Unit tests — the merge join algorithm, the rule engine operators (EQUAL, AMOUNT_TOLERANCE, FUZZY), hash computation,
bitfield operations. These are pure functions with no external dependencies — just Go test files with table-driven test cases.

Integration tests — testing the full ingestion pipeline against a real PostgreSQL instance (or a Docker test database). We test: parse a
real CSV file → insert to canonical_txn → verify recon_master is created correctly.

End-to-end scenario tests — upload a BANK file and an INTERNAL file with matching transactions → verify recon_master.status =
MATCH. Upload files with a ₹ 1 difference → verify rule engine marks it as PARTIAL_MATCH with AMOUNT_TOLERANCE.

We used testify/assert for assertions. Table-driven tests in Go let us test 20 scenarios with one function — just a slice of
input/expected pairs."

Q58: How would you write a unit test for the rule engine's AMOUNT_TOLERANCE operator?
SAY THIS: "I'd write a table-driven test like this:

tests := []struct {
name string
src int64
target int64
tolerance string
wantMatch bool
}{
{name: 'exact match', src: 50000, target: 50000, tolerance: 'FIXED:100', wantMatch: true},
{name: 'within fixed', src: 50000, target: 50050, tolerance: 'FIXED:100', wantMatch: true},
{name: 'exceeds fixed', src: 50000, target: 50200, tolerance: 'FIXED:100', wantMatch: false},
{name: 'within percent', src: 50000, target: 50900, tolerance: 'PERCENT:2', wantMatch: true},
{name: 'exceeds percent', src: 50000, target: 52000, tolerance: 'PERCENT:2', wantMatch: false},
{name: 'zero difference', src: 0, target: 0, tolerance: 'FIXED:0', wantMatch: true},
}
for _, tt := range tests {
[Link]([Link], func(t *testing.T) {
result := evaluateAmountTolerance([Link], [Link], [Link])
[Link](t, [Link], [Link])
})
}

Each test case is a struct. The loop runs every case. On failure, you see exactly which case failed."

Q59: What is mocking and when would you use it?

SAY THIS: "Mocking replaces a real dependency with a fake one that you control. For example, to test FileIngestionService
without a real database, you create a mock Repository interface — one that returns hardcoded data when called and records
which methods were called.

Use mocks when: the real dependency is slow (database, network), has side effects you don't want in tests (writing to S3, sending
Kafka messages), or is non-deterministic (current time, random).

Don't over-mock. In this project, the merge join is a pure function — no mocking needed, just test inputs and outputs directly. Only
mock when the real thing makes tests slow, fragile, or impossible to control."

Q60: How do you test that the retry mechanism works correctly?
SAY THIS: "I'd test it with a controlled failure scenario:

1. Create a test file with 3 batches (15000 rows at 5000 per batch)
2. Mock the repository to succeed on batch 1, fail on batch 2 with a specific error
3. Verify: rows_inserted = 5000 after the failure (first batch committed)
4. Now call the retry method with this raw_file
5. The retry should skip the first 5000 rows and start from row 5001
6. Verify: rows_inserted = 15000 after retry succeeds
7. Verify: the first 5000 rows are NOT duplicated in canonical_txn

The key assertion is no duplication — that's the whole point of the offset-based retry."

SECTION M — More Amazon Leadership Principle STAR Questions

Q61: Tell me about a time you took ownership beyond your assigned task. (Ownership)

SAY THIS (STAR): "Situation: I was assigned to build the SFTP polling worker — just the file detection part. During testing I noticed
the retry worker was re-processing failed files from row 0 every time, even if half the rows had already been inserted successfully.
This wasn't my task, but it was a serious bug — double inserts.

Task: Fix the retry offset issue even though it wasn't my assigned work.

Action: I traced through the ingestion code and found that rows_inserted was being updated only at file completion, not per batch. I
changed it to update after every successful batch commit. Then in the retry worker, I added the offset-based skip using rows_inserted
as the start position.

Result: The retry worker now resumes exactly where it left off. I flagged this to my team lead before making the change, explained
the impact, got approval, then implemented and tested it. Taking ownership of adjacent problems is faster than waiting for someone
else to discover the same issue."

Q62: Tell me about a time you had a disagreement with a teammate. (Earn Trust / Have Backbone)

SAY THIS (STAR): "Situation: A teammate proposed storing amounts as float64 to simplify the code — no need to convert from
rupees to paise everywhere.

Task: Decide the right approach for financial data storage.

Action: I respectfully pushed back with a concrete example: 0.1 + 0.2 in IEEE 754 floating point gives 0.30000000000000004.
In a reconciliation system, two records of ₹ 0.30 stored as separate floats might not be equal when compared. I wrote a small Go
program demonstrating this and shared it in our Slack. I proposed storing amounts as int64 in paise as the industry standard —
Stripe, Razorpay, and NPCI all do this.

Result: The team agreed to use paise. My teammate appreciated having a concrete demonstration rather than just 'floats are bad.'
The codebase now has a comment explaining why paise is used — for future team members."
 Q63: Tell me about a time you had to learn something new quickly. (Learn and Be Curious)

SAY THIS (STAR): "Situation: I was assigned to build the SFTP integration, but had never worked with SFTP or the pkg/sftp
library in Go before.

Task: Build a production-ready SFTP polling system in one sprint.

Action: I started by reading the pkg/sftp documentation and the SSH/SFTP RFC to understand the protocol. I set up a local SFTP
server using OpenSSH to test against. I built a minimal prototype first — just connect, list files, read one file. Then added the safety
filter (skip files < 1 minute old), error handling, and retry logic. I pair-programmed with a senior engineer for 30 minutes to review my
approach before building the full implementation.

Result: The SFTP worker was delivered in the sprint. I also documented the SFTP setup process for the team since no one had done
it before — turned my learning into a team resource."

Q64: Tell me about a time you delivered something with incomplete information. (Bias for Action)

SAY THIS (STAR): "Situation: We needed to process NPCI files but NPCI hadn't finalized their file format specification. We had a
draft spec but not the final one.

Task: Build the NPCI parser without a final spec, while still hitting the sprint deadline.

Action: I built the template-based parser system to be data-driven — the column mappings are in the database, not hardcoded. I
implemented the parser against the draft spec and loaded the draft template into the DB. I documented the assumption: 'template ID
3 is based on NPCI draft v0.3, update when final spec arrives.'

Result: The parser was done and working with the draft spec. When NPCI sent the final spec with 3 column changes, we just
updated the template in the DB — zero code changes. The upfront investment in a data-driven template system paid off exactly when
we needed it."

Q65: Tell me about a time you failed. What did you learn? (Learn from Mistakes)

SAY THIS (STAR): "Situation: I implemented the cache refresh without considering what happens during the refresh window.
Between the old cache being cleared and the new cache being populated, there was a ~100ms window where cache reads returned
nil.

Task: Fix production errors where nil cache was causing panics in the reconciliation worker.

Action: First fix was immediate — added nil checks before using cache data. Proper fix was to change the cache architecture:
instead of clear-then-reload, use atomic swap — load the new snapshot in a temporary variable, then atomically replace the
reference. The old snapshot is always valid until the moment of swap. No nil window.

Result: Zero-downtime cache refreshes. The learning: always think about the intermediate states during state transitions, not just the
before and after states."

Q66: Tell me about a time you simplified a complex problem. (Invent and Simplify)
 SAY THIS (STAR): "Situation: The initial design for tracking which systems had reported a transaction was a junction table —
transaction_system_presence(recon_key, system_id, arrived_at). Querying 'which systems are present for this
recon_key' required a JOIN.

Task: Make the reconciliation worker faster — it was doing many joins per polling cycle.

Action: I proposed replacing the junction table with a bitfield column on recon_master. Each system gets a fixed bit position —
INTERNAL=bit0, BANK=bit1, NPCI=bit2. Checking if BANK arrived is system_mask & (1<<1) != 0 — one bitwise AND, no join
needed. Setting all three systems present is system_mask = 0b111 — one integer update.

Result: Removed an entire table from the schema. Reconciliation queries got faster. The code is simpler — bitwise operations are
easier to reason about than JOIN conditions for this use case."

Q67: Tell me about a time you worked on a tight deadline. (Deliver Results)

SAY THIS (STAR): "Situation: A regulatory audit required that all reconciliation discrepancies from the last 90 days be traceable —
each mismatch needed a decision log explaining why it was MATCH or NO_MATCH. We had 2 weeks to implement this.

Task: Add a rule evaluation audit trail without breaking existing reconciliation logic.

Action: I prioritized: the core change was adding the rule_evaluations table and writing RuleEvaluation records after rule engine
runs. I did that first. Then I added the query API to retrieve evaluations by recon_key. I skipped the 'nice-to-have' features like
evaluation analytics dashboard — those weren't required for the audit. I shipped in 10 days, spent the last 4 days writing 90-day
backfill scripts for historical data.

Result: Passed the audit. The key was ruthless prioritization — delivered the exact requirement, nothing more."

SECTION N — "What If" Failure Scenarios (Flipkart, Swiggy, Amazon

SRE-style)
Q68: What happens if the PostgreSQL database goes down?
SAY THIS: "Three things happen simultaneously:

File processing fails — the ingestion worker gets a DB connection error. Since each batch is wrapped in a transaction, the in-progress
batch rolls back. rows_inserted is not advanced. The worker logs the error and marks the file as FAILED. RetryWorker will pick it
up when DB is back.

Reconciliation stops — the reconciliation worker's next poll will fail to query recon_master. The worker should have retry logic with
exponential backoff — it waits 5s, 10s, 20s between retries without hammering the dead DB.

SFTP polling continues reading files — but can't store them. This is where Kafka helps — instead of directly writing to DB, the SFTP
worker publishes a 'file arrived' event to Kafka. Kafka retains it durably. When DB recovers, the consumer picks up and processes all
queued events.

Critical: cache data is still in memory. Workers can still read config without hitting DB. The system degrades gracefully — no new
processing, but no crash."

Q69: What happens if an SFTP server goes down?

SAY THIS: "The SFTP worker polls every 60 seconds. If it can't connect to an SFTP source, it logs the error and tries the next source
in its list. It doesn't crash — one bad source doesn't affect processing of other sources.

For retry: the worker retries the connection on the next polling cycle (60 seconds later). If the SFTP server is down for hours, the files
just sit on the SFTP server. When the connection is restored, they're all picked up in the next poll. Banks typically retain files for 24-48
hours on their SFTP server.

The safety filter (skip files < 1 minute old) still works correctly when the server comes back — files that were there before the outage
are old enough to process immediately."

Q70: What happens if the service crashes and restarts mid-reconciliation?

SAY THIS: "The reconciliation worker doesn't maintain any in-memory state across cycles — it reads from DB, processes, writes to
DB, and that's it. On restart, the new instance reads recon_master and finds records still in PENDING. It picks them up and
processes normally.

The key design principle: no critical state exists only in memory. Everything is checkpointed in PostgreSQL. A crash means at most
one polling cycle (5 seconds) of delay, not data loss.

The one concern is partial batch writes during a reconciliation update — if the service crashes after updating pair_status_mask for 50
records but before updating the remaining 50. On restart, the 50 already-updated records are fine — they'll just be skipped. The 50
not yet updated will be picked up in the next cycle. Idempotency means running the reconciliation check twice produces the same
final result."

Q71: What if two workers try to reconcile the same transaction simultaneously?
SAY THIS: "This is prevented by using SELECT FOR UPDATE or row-level locks when claiming batches from recon_master. The first
worker to execute the SELECT locks those rows. The second worker either waits (blocking) or skips them (skip locked).

We use SKIP LOCKED — the second worker gets a different batch of rows and processes those instead. No waiting, no blocking.
Transactions that are already being reconciled are simply skipped by other workers. This is PostgreSQL's advisory lock pattern for
worker pools.

The result: multiple worker instances can reconcile different transactions truly in parallel, but no two workers ever process the same
transaction."

Q72: What if a file has 10 million rows and the server runs out of memory?

SAY THIS: "This is prevented by design — streaming in 5000-row chunks. The memory usage is bounded by the chunk size, not the
file size. Each 5000-row chunk is processed, committed to DB, and then garbage collected before the next chunk is loaded.

What does 5000 rows cost in memory? Each CanonicalTxn is roughly 200 bytes. 5000 × 200 = 1MB. Plus overhead from parsing,
maybe 3-5x = 3-5MB per batch. A server with 512MB of RAM can easily handle this.

If somehow the server does OOM-kill: the in-progress batch rolls back (DB transaction), rows_inserted stays at the last committed
value, and the RetryWorker picks it up from that offset. The file is processed in the next attempt."

SECTION O — Technology Comparison Questions

Q73: Why Go over Java for this project?

SAY THIS: "Three concrete reasons for this project specifically:

First, goroutines vs. Java threads. We have ~10 concurrent workers (polling at different intervals). In Java, each would be an OS
thread — ~1MB stack each, expensive context switches. In Go, goroutines start at 8KB. Running 10 or 1000 goroutines has virtually
no overhead difference.

Second, deployment. Go compiles to a single statically-linked binary. No JVM, no classpath, no dependency jars. Docker image is 10-
20MB. Java Spring Boot images are typically 200-500MB. For a service that might run on many small VMs, this matters.

Third, built-in tooling. go vet, go race (race condition detector), pprof profiling — all built-in, no external setup. For a financial
service where correctness is critical, the race detector finding potential data races at test time is invaluable."

Q74: Why PostgreSQL over MySQL?

SAY THIS: "For this project specifically, two reasons:

First, bitwise operations. pair_status_mask & 0x3 for checking pair status — PostgreSQL supports bitwise operators on integer
types natively and efficiently. Both work, but PostgreSQL's support is more mature.

Second, SELECT FOR UPDATE SKIP LOCKED — this is how the worker pool avoids processing the same file twice. MySQL added
SKIP LOCKED only in version 8.0. PostgreSQL has had it longer and with more mature semantics.

General reasons: PostgreSQL has better standards compliance, richer data types (native JSON, arrays, ranges), and MVCC is
implemented more cleanly. For financial data with complex queries and constraints, PostgreSQL is generally the safer choice."

Q75: Why Kafka over RabbitMQ?

SAY THIS: "Two main differences:

Kafka retains messages on disk for a configurable period (days/weeks). If our consumer is down for 2 hours, all messages are waiting
when it comes back — nothing lost. RabbitMQ removes messages once consumed; if no consumer is running, messages can be lost.

Kafka's consumer groups allow multiple instances of our service to share load AND allows different services to each read the full
stream independently. RabbitMQ distributes messages to consumers in round-robin — once consumed by one, it's gone for others.

For financial file events where durability and replay are critical — if we need to reprocess a file event from 3 days ago — Kafka is the
right choice. RabbitMQ is simpler and faster for task queues where durability is less critical."

Q76: Why AWS S3 over keeping files only on local disk?

SAY THIS: "Three reasons:

First, multi-instance deployment. If you run 3 instances of the service, and a file is stored on instance 1's local disk, instances 2 and 3
can't access it. S3 is a shared, durable store accessible by all instances.

Second, durability. S3 stores data across multiple availability zones — 99.999999999% durability (eleven 9s). Local disk fails with the
server. For financial transaction files that may need to be re-processed or audited months later, S3 is the right store.

Third, the abstraction is already there. We have a Storage interface. Local disk works in development for speed. S3 is swapped in
for production with a config change. The code doesn't change."

SECTION P — Security & Compliance Questions (Fintech focus)

Q77: How does your system protect financial data?
SAY THIS: "Five layers:

1. Encryption in transit — Kafka uses TLS with mutual certificate authentication. External DB connection uses
sslmode=require. SFTP itself encrypts the channel with SSH.

2. PGP file decryption — banks send files PGP-encrypted. We decrypt in memory using ProtonMail's go-crypto library before
parsing. The plaintext is never written to disk.

3. Least privilege DB access — the external DB connection only has SELECT permission. Even if our code had a bug that
tried to DELETE from the external DB, it would be rejected at the DB level.

4. No plaintext sensitive data in logs — we log recon_key (transaction ID) for debugging, but never log amounts or account
numbers. Logs should be safe to share with developers.

5. Audit trail — every rule evaluation decision is stored in rule_evaluations with timestamps. Every file processing step is
tracked in raw_file. This supports forensic investigation if there's ever a dispute."

Q78: What is SQL injection and does your code prevent it?

SAY THIS: "SQL injection is when user-supplied input is concatenated into a SQL query string, letting attackers run arbitrary SQL.
Example of vulnerable code: 'SELECT * FROM users WHERE name = ' + userInput. If userInput is '; DROP TABLE
users; --, the table gets dropped.

We use parameterized queries everywhere through pgx — never string concatenation. For example, [Link]('SELECT * FROM
canonical_txn WHERE recon_key = $1', reconKey). The $1 is a placeholder; pgx sends the query and the parameters
separately. The database treats the parameter as data, never as SQL syntax. This completely prevents SQL injection regardless of
what's in reconKey."

Q79: What is TLS and why is mutual TLS used for Kafka?

SAY THIS: "TLS (Transport Layer Security) encrypts communication between two parties. In standard TLS (like HTTPS), only the
server proves its identity — you verify the server's certificate. You, as client, are anonymous.

Mutual TLS means BOTH parties prove identity. The Kafka broker verifies our client certificate ([Link]), and we verify the
broker's certificate against our CA. This prevents two attacks:

1. A rogue Kafka broker intercepting our financial transaction events

2. An unauthorized service injecting fake file events into our Kafka topics

For financial systems, mutual TLS is required by most regulatory frameworks — it proves who sent which message."

SECTION Q — Monitoring, Logging & Observability (SRE-style

questions)
 Q80: How would you monitor this system in production?

SAY THIS: "Four areas to monitor:

File processing health:

Files in FAILED status older than 30 minutes (alert)

Average time from file arrival to COMPLETED status
Retry count increasing for a file (recurring failure)

Reconciliation health:

recon_master records stuck in PENDING for >1 hour

MISMATCH rate — if suddenly 50% of transactions are MISMATCH, something systemic changed
Records processed per second by reconciliation worker

System health:

DB connection pool utilization — if consistently >80%, need more connections

Goroutine count — if growing unboundedly, goroutine leak
Memory usage per service instance

Kafka lag:

Consumer group lag — how many messages are unprocessed. If lag is growing, our consumers are slower than producers.

Tools: Prometheus for metrics collection, Grafana for dashboards, PagerDuty for alerts on critical conditions."

Q81: What would you log in this system?

SAY THIS: "Log at these points with structured JSON logging:

File picked up from SFTP (filename, system, file_size)

File processing started/completed (file_id, rows_total, duration_ms)
Batch committed (file_id, batch_number, rows_inserted, cumulative_rows)
File FAILED (file_id, error_message, rows_inserted_at_failure)
Reconciliation cycle completed (records_matched, records_mismatched, duration_ms)
Rule evaluation result (recon_key, rule_id, decision) — at DEBUG level, not INFO
Cache refresh completed (cache_name, records_loaded, duration_ms)

What NOT to log:

Account numbers, card numbers, user PII — even in debug logs

Full transaction amounts in bulk — log counts, not individual values
Raw file contents — too much data, and may contain sensitive info

Structured JSON logging means every log line is parseable — tools like Elasticsearch can query 'show me all files that failed with this
specific error' without regex parsing."

Q82: How would you debug a situation where reconciliation is stuck in PENDING for hours?
 SAY THIS: "Step-by-step diagnosis:

1. Check which systems are present: SELECT recon_key, system_mask, pair_status_mask, status FROM
recon_master WHERE status = 'PENDING' AND updated_at < now() - interval '1 hour'

2. If system_mask shows only BANK (no INTERNAL): InternalSyncWorker isn't pulling this transaction. Check
internal_pull_state table — is the cursor advancing? Check InternalSyncWorker logs for errors.

3. If both systems present but still PENDING: reconciliation worker isn't processing it. Check reconciliation worker logs. Check if
the worker is alive. Check DB connections.

4. If pair_status_mask shows OPEN for a pair even with both systems: the merge join isn't matching. Maybe recon_key formats
differ between systems — BANK sends UPI123 but INTERNAL stores upi123 (case sensitivity). Check canonical_txn for
both systems with that recon_key.

5. Nuclear option: add temporary debug logging to the reconciliation worker to print what keys it's processing and the result of
each comparison."

SECTION R — Computer Science Fundamentals (Google, Microsoft)

Q83: Explain ACID properties. Which ones does your system rely on most?

SAY THIS: "ACID stands for Atomicity, Consistency, Isolation, Durability.

Atomicity — either all operations in a transaction succeed, or none do. We rely on this heavily: inserting 5000 canonical transactions
+ updating rows_inserted + upserting recon_master records all happen in one transaction. If any step fails, all roll back.

Consistency — the DB moves from one valid state to another. Our constraints enforce this: recon_key must have a valid system_id,
foreign keys between tables are enforced.

Isolation — concurrent transactions don't see each other's uncommitted changes. When two workers run simultaneously, each sees
a consistent snapshot of the DB. One worker updating pair_status_mask doesn't affect what another worker reads until committed.

Durability — once committed, data survives crashes. PostgreSQL writes to WAL (Write-Ahead Log) before confirming commit. Even
if the server crashes a millisecond after commit, the data is safe.

We rely on Atomicity and Durability most — they're the foundation of the checkpoint-and-retry pattern."

Q84: What is the difference between concurrency and parallelism?

SAY THIS: "Concurrency is about dealing with multiple things at once — structuring a program to handle multiple tasks. Parallelism is
about doing multiple things simultaneously — actually executing at the same time on different CPU cores.

Concurrency doesn't require parallelism: a single-core CPU can be concurrent (switching between tasks), but not parallel.

In this project: our Go service is concurrent — it has multiple goroutines for reconciliation worker, SFTP worker, retry worker, cache
refresh. On a single-core machine, they'd time-slice. On a multi-core machine (typical production server), Go's runtime schedules
goroutines across all cores — true parallelism.

The DB batch insert is serial within a batch (one query for 5000 rows), but different files can be processed in parallel by different
service instances."

Q85: What is a deadlock? Can your system have one?

SAY THIS: "A deadlock is when two processes each hold a resource the other needs, and both wait forever. Example: Process A
holds Lock 1 and waits for Lock 2. Process B holds Lock 2 and waits for Lock 1. Neither can proceed.

In a database: Transaction A updates recon_master row X then tries to lock canonical_txn row Y. Transaction B updates
canonical_txn row Y then tries to lock recon_master row X. Deadlock.

Prevention in this system: we always acquire locks in a consistent order — canonical_txn is always updated before recon_master
within a transaction, never the reverse. PostgreSQL also has deadlock detection — if it detects a deadlock, it picks one transaction,
rolls it back, and lets the other proceed. Our code handles this by catching the deadlock error and retrying the transaction."

Q86: What is connection pooling and why is it critical here?

SAY THIS: "Creating a new database connection is expensive — it involves TCP handshake, authentication, SSL negotiation —
typically 20-50ms. If every DB query opened and closed a new connection, processing 5000 rows would cost 5000 × 50ms = 250
seconds just in connection overhead.

Connection pooling maintains a set of pre-opened connections. When your code needs to query the DB, it borrows a connection from
the pool, uses it, and returns it. Borrowing takes microseconds, not milliseconds.

pgx's pool manages min/max pool size, idle timeout, health checks. When our service starts, pgx opens a minimum number of
connections. Under load, it opens more (up to max). Connections idle for too long are closed. If all connections are in use, new
requests wait for one to free up — this prevents overwhelming the DB with too many simultaneous connections."

Q87: What is the difference between a process and a thread? Where do goroutines fit?
SAY THIS: "A process is an isolated program with its own memory space. Processes communicate via IPC (pipes, sockets, shared
memory). Creating a process is expensive — hundreds of milliseconds.

A thread is a unit of execution within a process — multiple threads share the same memory space. Threads communicate by
reading/writing shared variables (which is why you need mutexes). Creating a thread costs ~1MB of stack and OS overhead.

A goroutine is Go's userspace thread — managed by Go's runtime, not the OS. Goroutines start with 8KB stack (grows dynamically).
Creating a goroutine costs microseconds. The Go runtime multiplexes many goroutines onto fewer OS threads using M:N scheduling.

In this project: everything runs in one process. Multiple goroutines handle different workers. They share memory (the cache's
[Link] is accessed by all goroutines) but coordinate safely through atomic operations and channels."

Q88: What is an HTTP status code? What would the API return in different scenarios?

SAY THIS: "HTTP status codes indicate the result of a request. Key ones for this system's API:

200 OK — successful GET, file status retrieved

201 Created — file upload successful, returns file_id
202 Accepted — report generation started (async), returns job_id
400 Bad Request — invalid file format, missing required fields
404 Not Found — requested recon_key or file_id doesn't exist
409 Conflict — file already exists and is being processed
422 Unprocessable Entity — file format is valid JSON/CSV but semantic error (unknown system_id)
500 Internal Server Error — unexpected server error (DB down, OOM)
503 Service Unavailable — service is shutting down, stop sending requests

For async operations like report generation: return 202 Accepted immediately with a job_id. The client polls GET
/reports/{job_id} which returns 200 with status='processing' until done, then 200 with status='completed' and download URL."

SECTION S — Edge Cases & Tricky Questions

Q89: What if the recon_key is the same but it's actually a different transaction? (Hash collision vulnerability?)

SAY THIS: "Good question. SHA256 has a collision resistance of 2^128 — practically impossible to find two different inputs with the
same hash. But the recon_key collision is a more realistic concern — what if two different transactions have the same UPI reference
number?

In theory, UPI reference numbers (RRN) are unique per transaction. But bugs happen. If two different transactions share a recon_key,
our system would compare their hashes. If the amounts also coincidentally match, we'd falsely call it a MATCH.

Mitigation: validate recon_key uniqueness within a system during ingestion. If the same recon_key appears twice in a BANK file, flag
it as a data quality issue rather than silently deduplicating. The is_latest flag handles intentional corrections, not accidental
duplicates."
 Q90: What if the rule engine has a bug and marks a genuine mismatch as MATCH?

SAY THIS: "This is a compliance risk. The mitigation layers:

First, rules are in the DB with an active flag — a rule can be disabled without code deployment. If a bug is found, disable the rule
immediately.

Second, rule_evaluations table stores the full evaluation — src field values, target field values, each condition's result. A
compliance team can query: 'show me all transactions where rule X was applied and the decision was MATCH, with the actual field
values.' This is the audit trail.

Third, there should be a manual review queue for any transaction where rules were applied (rule_applied = true). A human
periodically spot-checks these to catch systematic bugs.

Fourth, the system should have canary rules — rules with known correct answers on test data — that run in parallel to detect rule
engine regressions."

Q91: How do you handle timezone differences between systems?

SAY THIS: "Timezone differences are a common source of TIME_RANGE false mismatches. A transaction that happens at 11:30 PM
IST might be recorded as 6:00 PM UTC by one system and 11:30 PM IST by another — they're the same moment in time but look 5.5
hours different.

The fix has two parts: First, all timestamps are normalized to UTC during canonicalization. The template column transformer includes
timezone conversion — if a bank sends IST timestamps, the transformer converts to UTC before storing in canonical_txn.
Second, the TIME_RANGE operator compares timestamps after normalization — both are UTC, so the difference is the actual wall-
clock difference between when the systems recorded the event.

The tolerance in TIME_RANGE (e.g., 60 seconds) accounts for legitimate clock drift between systems, not timezone differences —
those are eliminated at ingestion time."

Q92: What is idempotency and why is it critical for this system?

SAY THIS: "Idempotency means running the same operation multiple times produces the same result as running it once. A function
f(x) is idempotent if f(f(x)) = f(x).

It's critical here because failures happen, and recovery means re-running operations. If re-running causes duplicate data, you've
made things worse.

Our ingestion is idempotent via the is_latest flag — re-ingesting the same file marks old records as not-latest and inserts fresh
copies. The reconciliation worker is idempotent — running it on already-MATCH records finds them MATCH again and updates the
same values. The retry worker is idempotent — resuming from rows_inserted offset means rows already inserted won't be re-
inserted.

Without idempotency: a Kafka consumer that processes a message and crashes before committing the offset would receive the
message again on restart. If the handler isn't idempotent, you get double records."
 PART 5 — THINGS TO NEVER SAY IN
INTERVIEWS
Don't Say Say Instead
"I just used what was already set
"I chose/contributed to this decision because..."
up"
"I don't know how that part works" "That part was handled by X, but what I understand is..."
"It was easy" "The straightforward part was X, the challenging part was Y"
"I copy-pasted from StackOverflow" "I referenced documentation and adapted it for our use case"
"It's basically like [simple analogy]
Use the analogy, then say "specifically in our case..."
but more complex"
"Testing that specific edge case is a good point — we covered
"We didn't test that"
X,Y,Z. For that scenario, we relied on Z"

PART 6 — QUICK REFERENCE CHEAT SHEET

System Numbers to Remember
What Value
Reconciler polls every 5 seconds
Internal Sync polls every 10 seconds
SFTP polls every 60 seconds
File streaming chunk size 5000 rows
Cache TTL 2 hours
File lease window 10 minutes
pair_status_mask bits per pair 2 bits
MATCH bit value 01
MISMATCH bit value 10
Hash algorithm SHA256

Key DB Tables and What They Do

Table One-liner
canonical_txn Normalized transactions from all systems, with hash and is_latest
recon_master Reconciliation state per transaction (system_mask, pair_status_mask, status)
raw_file File processing state (status, rows_inserted, lease, retry_count)
file_template File format definitions (delimiter, has_header, date_format)
file_template_column Column mappings (index → field name → data type → transform)
recon_rules Business rules for evaluating mismatches (conditions_json)
system_registry System definitions with bit positions
recon_pair_ref Which systems form pairs and at which bit offset
source_config SFTP source credentials and paths
rule_evaluations Audit log of every rule engine decision
internal_pull_state Last synced timestamp per subsystem (cursor for InternalSyncWorker)

Four Operators in Rule Engine

Operator What it checks Example use
 Operator What it checks Example use
EQUAL Exact match Transaction status must be the same
AMOUNT_TOLERANCE Difference within X paise or X% Allow ±2% for rounding differences
TIME_RANGE Timestamp within N seconds Allow ±60s for system clock drift
FUZZY String similarity Merchant name slightly different spellings

Four Caches and What They Store

Cache Stores Used By
TemplateCache File format + column definitions File parser, ingestion
ReconMetaCache Systems, pairs, SFTP sources Reconciler worker, SFTP worker
RuleCache Rules grouped by source system Rule engine
MasterCache Worker configs (table names, batch sizes) Worker factory

Status Values to Know

Field Values
raw_file.status RECEIVED → INPROGRESS → COMPLETED / FAILED
recon_master.status PENDING → MATCH or NO_MATCH
pair_status_mask 2-bit value 00=OPEN, 01=MATCH, 10=MISMATCH, 11=PENDING
rule_evaluation.final_decision MATCH, PARTIAL_MATCH, NO_MATCH

Good luck. You built this system — you know it better than any interviewer ever will.