Table of Contents

Edumate Backend — Document Management System (DMS)

Module: documents (/api/v1/documents) — a new module inside the existing
edumate-auth monolith that manages the full document lifecycle for loan
applications: requirement resolution, secure S3-backed uploads, virus scan + OCR,
verification, and versioned resubmissions.
Design goal: Reuse — no new architectural patterns. The DMS slots into the
existing routes → controller → service → dto layering, RBAC
(resource:action:scope), partner-scope filtering, in-process workers,
email_outbox, audit_logs, and the S3 client already in the codebase.
Last drafted: 2026-07-10 · Stack: [Link] 20 · Express · TypeScript · Prisma 7 ·
PostgreSQL · Redis · AWS S3 (ap-south-1)

1. Why the DMS Exists

Today, document tracking on a loan application is effectively free text. There is no
canonical record of which document a specific person submitted, what version it is, whether
it passed verification, or who accessed it. For an education-lending platform handling
identity proofs, income documents, and bank statements across multiple countries and
lenders, that is both an operational gap and a compliance risk.
The DMS closes that gap by introducing a proper document domain that:
• Knows, for any application, which documents are required, for which party, and
why (driven by a catalog of rules rather than manual data entry).
• Stores every uploaded file securely in S3 with metadata, integrity hashes, and
version history in Postgres.
• Runs each upload through virus scanning, OCR, and validation before a human
ever sees it.
• Enforces object-level access control and writes an audit trail for every verify,
reject, and download.
The module is designed to be additive: existing loan-application, disbursement, and
commission flows are untouched.

2. Core Concept — Three Separate Layers

The single most important design decision is to keep three distinct concepts separate.
Collapsing them is what makes document systems unmaintainable at scale.
 Layer Table What it is Changes how often
Definition document_types The catalog. “What Rarely
(+ is a PAN Card, in
document_type_ru what format, at
les) what quality, and
when is it required.”
Reference data.
Requirement document_require A resolved obligation Per application
ments for one application
and one party.
“Application #4521,
PAN Card required
for Co-Applicant 1.”
Computed from the
catalog rules.
Instance documents An actual uploaded Per upload /
file — a specific resubmission
person’s file for a
specific
requirement,
versioned.

A requirement is derived (the resolver reads the catalog rules against the application’s
country, course, lender, and loan product). An instance is created when someone uploads.
Keeping them separate means a requirement can exist as “still pending” with zero files,
carry multiple file versions over its lifetime, and roll up a clean status without any of that
logic leaking into the file record.

3. Runtime Architecture — Control Plane vs Data Plane

Three components, three distinct roles. File bytes never pass through the Node process
— not on upload, not on download.
┌──────────────────────────────────────────────┐
Student portal ───▶│ Node API (/api/v1/documents) │
CONTROL PLANE
Partner / API │ auth · validate · sign URLs · state · RBAC │
(metadata + decisions)
Ops console └───────────────┬──────────────────────────────┘
│
┌───────────────────────┼───────────────────────┐
▼ ▼ ▼
┌───────────┐ ┌────────────┐
┌─────────────────┐
│ S3 │◀─direct──│ (clients │ │ PostgreSQL
│ STATE
│ ap-south-1│ upload/ │ bypass │ │ metadata +
│ (source of truth)
│ bytes │ download│ the API) │ │ status +
audit │
└───────────┘ └────────────┘
└─────────────────┘
▲ ▲
│ fetch for scan/OCR │
claim jobs
└──────────────── Document Processing Worker ──────┘
DATA-PLANE PROCESSING
(virus scan · checksum · OCR)

• Node = control plane. Authenticates, authorises, validates against catalog rules,

writes metadata, and issues short-lived pre-signed S3 URLs. It never streams the file
body during normal upload/download.
• S3 = data plane. Holds raw bytes. Clients PUT/GET directly against pre-signed
URLs. This keeps the API stateless and horizontally scalable — a burst of large PDF
uploads never touches the app tier.
• Postgres = source of truth. Holds the file’s metadata, its lifecycle status, and the
audit trail. S3 does not know or care whether a file is “verified”; all state and logic
live here.
• Worker = data-plane processing. The only server-side component that pulls file
bytes, and only asynchronously, after upload, to scan and extract.

4. Database Schema
PostgreSQL via Prisma, following existing conventions: Int autoincrement PKs,
snake_case tables, is_active/is_deleted where relevant, audit timestamps, and a
partner_id bridge on scoped tables so the existing [Link] filter works
unchanged.
4.1 Entity relationships

Document module entity relationships

4.2 Table summary & lifecycle columns

Key status / lifecycle
Table Purpose columns Delete semantics
document_types Catalog of document is_active, Soft (is_deleted)
definitions (the is_deleted
~143-doc master)
document_type_ru Resolver rules per priority = Soft
les type (party, country, MANDATORY/CON
 Key status / lifecycle
Table Purpose columns Delete semantics
course, lender, DITIONAL/OPTION
product, priority, AL
stage)
document_require Resolved obligation status = Soft
ments per application + REQUESTED →
party UPLOADED →
IN_REVIEW →
VERIFIED /
REJECTED /
WAIVED
documents Actual uploaded file status, Soft (compliance
instance (versioned) scan_status, preservation)
ocr_status,
version,
is_current
document_access_ Append-only action = VIEW / Never deleted
logs download/view DOWNLOAD /
audit URL_ISSUED
Design note — versioning: a resubmission never overwrites. It inserts a new
documents row with version + 1 and sets the previous row’s is_current =
false. This preserves the full history of what was submitted, rejected, and re-
submitted.
Full field lists are in Appendix A.

5. Module Map & API Surface

Mounted with a one-line addition in src/[Link]:
[Link]("/documents", documentsRouter);

The module follows the standard routes → controller → service layering with a
dto/ folder of Zod schemas. Per-route guards compose exactly as elsewhere:
requireAuth → requirePermission([...]) → validate({...}) → controller.

# Endpoint Guard (permission) Purpose

1 POST document:create Resolver — reads
/documents/requi the application’s
rements/resolve country/course/len
der_id/loan_prod
uct_id against
document_type_ru
# Endpoint Guard (permission) Purpose
les and creates
document_require
ments rows
2 GET document:read Returns the
/documents/requi (:org) requirement
rements? checklist for a case,
loan_application with each
_id=
requirement’s
current status
3 POST document:create Manually add a
/documents/requi document type for a
rements party (override the
resolver)
4 PATCH document:verify Waive a
/documents/requi requirement with a
rements/:id/waiv reason
e
5 POST document:upload Validate format/size
/documents/uploa (:org/:own) vs document_type,
d-url insert a PENDING
documents row,
return a short-lived
pre-signed PUT URL
+ document_id
6 POST document:upload Confirm upload
/documents/:id/c finished → status
omplete UPLOADED →
enqueue scan/OCR
7 GET document:read Object-level RBAC
/documents/:id/f (:org/:own) check → short-lived
ile pre-signed GET
URL; every call
logged to
document_access_
logs
8 POST document:verify Mark document +
/documents/:id/v requirement
erify VERIFIED
9 POST document:verify Reason → REJECTED
/documents/:id/r → queue
eject resubmission email
via email_outbox
5.1 Example request/response — POST /documents/upload-url
// Request
{
"requirement_id": 812,
"file_name": "pan_card.pdf",
"mime_type": "application/pdf",
"size_bytes": 384210
}

// Response (201)
{
"document_id": 1904,
"upload_url": "[Link]
[Link]/...", // expires ~5 min
"s3_key": "app-4521/req-812/v1/pan_card.pdf",
"expires_in": 300
}

The client then issues a direct PUT to upload_url with the file body, and finally calls POST
/documents/1904/complete.

6. Authorization (RBAC) & Scoping

The DMS reuses the existing resource:action[:scope] permission model and match
order verbatim. New permission codes:
document:read document:read:org document:read:own
document:create document:upload document:upload:org
document:upload:own
document:verify document:delete

6.1 Role grants

Role Grants Effect
super_admin *:* Full bypass
Admin document:* Full, but restricted to
assigned partners (see
below)
Partner document:read:org, View + upload only on its
document:upload:org own leads’ documents
API_Partner document:create:org, Vendor key — submit
document:upload:org, documents for its own leads
document:read:org via API
Student user document:read:own, Only their own documents
document:upload:own
6.2 Partner scoping (the important part)
Because document_requirements carries partner_id, the existing src/lib/partner-
[Link] logic applies with no new code:

• A plain Admin sees only requirements/documents whose partner_id is in their

assigned set (assigned_admin_user_id = self). List queries add
where.partner_id IN (assigned ids); single-record reads call
assertPartnerInScope(...) → 403 if out of scope.
• super_admin, L1/L2, and API-key principals are unrestricted (subject to :org
pinning for partner keys).
• Partner-portal users are pinned to their own partner_id via the :org scope.

6.3 Object-level checks on file access

GET /documents/:id/file performs a per-object check before signing a URL: it resolves
the document → requirement → partner_id and confirms the caller is in scope (or owns it
via :own). This prevents S3-key guessing — a caller can never obtain a signed URL for a
document outside their scope. Every denial writes an audit_logs row (required vs held
permissions), consistent with existing behaviour.

7. Upload Flow (Node ↔ S3 ↔ Postgres)

Upload flow sequence

Why pre-signed URLs matter: in step 4 the file goes straight to S3 using a URL scoped to
exactly one key for a few minutes. Fifty partners uploading 20 MB PDFs never touch the
Node process — S3 absorbs it. The app tier handles only small JSON requests, so it stays
stateless and scalable. This is the same reasoning behind moving off a single Lightsail
instance later: keep the app tier out of the byte path from day one.
Download is the mirror image: the client asks the API, the API runs the object-level RBAC
check, and returns a short-lived pre-signed GET URL (or streams via a JWT-guarded route,
mirroring the existing GET /:id/invoice-file pattern — both are viable; pre-signed is
preferred at volume).
8. Admin Journey — End to End
Admin journey
Stage-by-stage, mapped across UI, API, and data:
1. Select case. UI: click a case in the applications list. Backend: existing GET
/loanapplications/:id + its linked contact (student, co-applicants, guarantor).
The admin sees the parties. No new endpoint.
2. Checklist auto-loads. First visit runs POST
/documents/requirements/resolve; thereafter GET
/documents/requirements. The resolver matches document_type_rules
against the application’s country/course/lender/product and writes
document_requirements rows (status = REQUESTED) per party. The admin can
add or waive — never a blank slate.
3. Request sent. The service inserts email_outbox rows; the existing email worker
notifies the student/partner. Admin’s work pauses here.
4. Upload. Student/partner uses the upload flow in §7. Files land in S3; the worker
scans, checksums, OCRs, and validates; status moves REQUESTED → UPLOADED →
IN_REVIEW. API-key uploads are automatically captured by the existing
apiUsageLog middleware into api_request_logs.
5. Review. The checklist now shows “In Review”. GET /documents/:id/file
returns a pre-signed URL (logged to document_access_logs) alongside
ocr_data, so the admin verifies extracted values instead of re-typing.
6. Verify / Reject. verify marks the requirement VERIFIED. reject with a reason
moves it to REJECTED, queues a resubmission email, and the next upload creates a
version + 1 row (old is_current = false). Each action writes to audit_logs,
mirroring the commission L1/L2 audit trail.
7. Complete. When every mandatory requirement is VERIFIED, the application’s
document stage is complete and downstream stages (credit assessment /
disbursement) can proceed.
The admin only truly acts at two points — defining what’s needed (step 2) and reviewing
(steps 5–6). Everything between is automated.

9. Document Lifecycle — State Machine

A single lifecycle spanning the requirement and its current document instance. Invalid
transitions are rejected at the service layer (no free-text status).
Document lifecycle state machine
State Meaning Who moves it
REQUESTED Required, nothing Resolver / admin
uploaded yet
UPLOADED File in S3, awaiting Client (/complete)
processing
SCANNING Worker scanning + Worker
OCR
IN_REVIEW Passed automated Worker
checks, awaiting
human
VERIFIED Accepted (terminal) Admin
REJECTED Rejected with reason → Admin / worker
resubmit
WAIVED Not required for this Admin
case (terminal)
10. Background Worker — Document Processing
A new in-process worker started in src/[Link], following the exact pattern of the
existing programs-fetch worker: safe to run on every instance because it claims rows
atomically.
claim: UPDATE documents
SET status = 'SCANNING', claimed_at = now()
WHERE id = (SELECT id FROM documents
WHERE status = 'UPLOADED' AND is_deleted = false
ORDER BY uploaded_at LIMIT 1
FOR UPDATE SKIP LOCKED)
RETURNING *;

process: fetch bytes from S3

→ virus scan (ClamAV / S3-integrated) → scan_status
→ sha256 checksum (integrity)
→ OCR if is_ocr_compatible → ocr_data
→ run document_type.validation_rules
→ status = IN_REVIEW (or REJECTED on infection / hard
validation failure)

notify: INSERT email_outbox row → admin "N documents ready for

review"

For MVP the documents table itself acts as the queue (poll by status = 'UPLOADED'). If
independent retry/backoff becomes necessary, introduce a dedicated
document_processing_jobs table (QUEUED → RUNNING → SUCCESS/FAILED)
mirroring program_fetch_jobs — same philosophy, isolated retries.

11. S3 Storage Design

• Bucket: a new application-documents bucket in ap-south-1 (separate from
application-invoices), reusing the existing @aws-sdk/client-s3 setup.
• Key layout:
app-{loan_application_id}/req-{requirement_id}/v{version}/{file_n
ame} — human-debuggable, collision-free across versions.
• Versioning: enabled on the bucket so an accidental overwrite or delete is still
recoverable at the object level.
• Encryption: SSE-KMS at rest, TLS in transit. For high-sensitivity types (identity,
income), consider application-level envelope encryption so raw S3 access alone
never yields plaintext.
• Access: exclusively via short-lived pre-signed URLs (upload and download). No
public access; no bucket-level listing exposed to the app.
12. Notifications
No new notification infrastructure. The DMS inserts rows into the existing email_outbox,
and the existing email worker (token-bucket rate limit + circuit breaker + backoff) delivers
them:
• Requirement created → notify student/partner to upload.
• Document rejected → notify with the reason and a resubmission link.
• All mandatory verified → notify the assigned admin that the case is document-
complete.
Recipients are resolved by role from the DB, consistent with the commission-settlement
notification approach.

13. Zoho CRM Integration (Deferred)

Zoho is intentionally out of scope for the first phases; the DMS is the source of truth for
documents. When integration lands, it reuses the existing transactional-outbox + trigger +
LISTEN/NOTIFY infrastructure:
• A trigger on document_requirements status changes enqueues a rollup into
zoho_sync_outbox on a new channel (e.g. zoho_sync_document_rollup).
• The rollup updates the corresponding loan_applications document-summary
fields on the Zoho Deal (required / submitted / pending counts + overall status).
• Files are never pushed to Zoho — only status. The last_synced_hash loop-
guard and app.zoho_origin session flag apply unchanged.

14. Production Concerns & Disaster Recovery

Directly informed by past incidents (accidental migrate:fresh and seeder runs against
the wrong database):
• Soft delete everywhere in the DMS. documents and document_requirements
use is_deleted; hard deletes only via a scheduled retention job driven by
document_type retention policy, never a direct call.
• S3 versioning ON so byte-level recovery survives accidental overwrite/delete.
• Migration guardrails. A NODE_ENV=production guard that refuses
migrate:fresh and seeders outright; seeders kept fully separate from migrations;
a confirmation gate on the production environment.
• Audit split, matching existing conventions. State changes (verify/reject/waive) →
audit_logs. High-volume file access (view/download) →
document_access_logs (the api_request_logs philosophy), so the security-
audit table stays signal-rich.
 • Malware scanning is mandatory before any document reaches IN_REVIEW — this
is user-uploaded content on a financial platform.
• GDPR / data residency. document_types carry data_sensitivity and a
retention policy; EU-resident data may require region-pinned buckets and a right-
to-erasure path (soft-delete → scheduled hard-delete → retained audit record).

15. Rollout Phases

Incremental, each phase independently shippable:

Phase Scope Outcome

1 documents table + S3 pre- Real file storage replaces
signed upload/download + free text
basic status
2 document_types + Per-application
document_type_rules + requirements auto-
resolver generate; manual entry
gone
3 Verification workflow + State machine, ops review,
audit + object-level RBAC scoped access
hardening
4 Processing worker: virus Automated pre-checks +
scan, OCR, validation rules extracted data on review
5 Scale + compliance: Production-grade, multi-
dedicated job queue, CDN, region, CRM-synced
region-pinning, DR drills,
Zoho rollup

Appendix A — Prisma Schema Sketch

Illustrative; field names/types to be finalised against prisma/[Link]
conventions.
model DocumentType {
id Int @id @default(autoincrement())
code String @unique
name String
display_name String
description String?
document_category String
accepted_formats String // e.g. "pdf,jpg,png"
max_file_size_mb Int
min_resolution_dpi Int?
is_ocr_compatible Boolean @default(false)
 extractable_fields Json?
validation_rules Json?
gdpr_relevant Boolean @default(false)
data_sensitivity String @default("Medium") // Low/Medium/High
retention_months Int?
is_active Boolean @default(true)
is_deleted Boolean @default(false)
created_at DateTime @default(now())
updated_at DateTime @updatedAt

rules DocumentTypeRule[]
requirements DocumentRequirement[]
@@map("document_types")
}

model DocumentTypeRule {
id Int @id @default(autoincrement())
document_type_id Int
applies_to_party String //
STUDENT/CO_APPLICANT/GUARANTOR/ANY
required_for_countries Json? // null = all
required_for_courses Json?
required_for_lender_ids Json?
required_for_loan_product_ids Json?
min_loan_amount Decimal?
priority String @default("MANDATORY") //
MANDATORY/CONDITIONAL/OPTIONAL
document_stage String @default("APPLICATION")
waiver_conditions String?
created_at DateTime @default(now())
updated_at DateTime @updatedAt

document_type DocumentType @relation(fields: [document_type_id],

references: [id])
@@map("document_type_rules")
}

model DocumentRequirement {
id Int @id @default(autoincrement())
loan_application_id Int
contact_id Int
partner_id Int // scope bridge —
reused by [Link]
document_type_id Int
party String //
STUDENT/CO_APPLICANT_1/2/3/GUARANTOR
is_mandatory Boolean @default(true)
status String @default("REQUESTED")
waived_reason String?
requested_by_user_id Int?
 resolved_at DateTime @default(now())
is_deleted Boolean @default(false)
created_at DateTime @default(now())
updated_at DateTime @updatedAt

document_type DocumentType @relation(fields: [document_type_id],

references: [id])
documents Document[]
@@unique([loan_application_id, document_type_id, party])
@@index([partner_id])
@@map("document_requirements")
}

model Document {
id Int @id @default(autoincrement())
requirement_id Int
uploaded_by_user_id Int?
uploaded_via_api_key_id Int?
s3_bucket String
s3_key String
file_name String
mime_type String
size_bytes Int
sha256 String?
version Int @default(1)
is_current Boolean @default(true)
status String @default("PENDING")
scan_status String @default("PENDING") //
PENDING/CLEAN/INFECTED/ERROR
ocr_status String @default("PENDING") //
PENDING/DONE/SKIPPED/FAILED
ocr_data Json?
rejection_reason String?
verified_by_user_id Int?
verified_at DateTime?
claimed_at DateTime?
is_deleted Boolean @default(false)
uploaded_at DateTime?
created_at DateTime @default(now())
updated_at DateTime @updatedAt

requirement DocumentRequirement @relation(fields: [requirement_id],

references: [id])
access_logs DocumentAccessLog[]
@@index([status])
@@index([requirement_id, is_current])
@@map("documents")
}

model DocumentAccessLog {
 id Int @id @default(autoincrement())
document_id Int
user_id Int?
api_key_id Int?
action String // VIEW/DOWNLOAD/URL_ISSUED
ip_address String?
request_id String?
created_at DateTime @default(now())

document Document @relation(fields: [document_id], references: [id])

@@index([document_id])
@@map("document_access_logs")
}

Appendix B — Permission Seed

Add to prisma/[Link] alongside existing permission seeding:
document:read document:read:org document:read:own
document:create
document:upload document:upload:org document:upload:own
document:verify
document:delete

Role → permission grants:

Role Document permissions

super_admin (covered by *:*)
Admin document:read, document:create,
document:upload, document:verify,
document:delete — partner-scoped
commission_reviewer / inherit Admin document perms
commission_approver (unrestricted)
Partner document:read:org,
document:upload:org
API_Partner document:read:org,
document:create:org,
document:upload:org
Student user role document:read:own,
document:upload:own

This document describes a proposed module. The authoritative reference on merge will be
prisma/[Link] and the module source under src/modules/documents/.