0% found this document useful (0 votes)
23 views17 pages

FastAPI File Storage Design

The document outlines a design for a FastAPI application that handles file uploads and metadata storage using PostgreSQL and Azure Blob Storage. It details the architecture, database schema, API endpoints, and best practices for managing risk records and their associated attachments, including generating downloadable Excel files and ZIP packages. The document also provides code examples for implementing the functionality, including file uploads, metadata storage, and data export.

Uploaded by

Reddy Venkata
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
23 views17 pages

FastAPI File Storage Design

The document outlines a design for a FastAPI application that handles file uploads and metadata storage using PostgreSQL and Azure Blob Storage. It details the architecture, database schema, API endpoints, and best practices for managing risk records and their associated attachments, including generating downloadable Excel files and ZIP packages. The document also provides code examples for implementing the functionality, including file uploads, metadata storage, and data export.

Uploaded by

Reddy Venkata
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

FastAPI File Storage Design

I have a frontend where I need to fill in attributes, and there is a column called supporting
documentation. User will upload a document, multiple attachments they'll upload it, okay? And we
need to make sure to save it properly, and there is a download option, so when they download it, it
should download all the attributes also, along with that attachment also should come out as in the
Excel. Okay? How do we, how can we do that one using the Python? I'm using FastAPI as the
backend tool, and backend database PostgreSQL. How can I design this one properly, able to store
it? And, and I'm using Azure file storage to store the files for each of the um each of the risk ID. So
user will upload the, they'll upload the attributes and attach the attachments in the attributes,
that's what they will do. How, how can I implement that logic? Can you advise?

You can design this cleanly by separating metadata (attributes) and files (attachments). The
metadata stays in PostgreSQL, and the actual files go to Microsoft Azure Azure File Storage / Blob
Storage. Your FastAPI backend acts as the orchestrator.

Below is a recommended architecture used in production systems.

1️⃣ Overall Architecture


Frontend → FastAPI API → PostgreSQL (attributes) + Azure Storage (files)

Flow:

1. User fills attributes and uploads files.

2. Backend:

Saves attributes in PostgreSQL

Uploads files to Azure storage


Stores file metadata (filename, path, risk_id) in database.
3. When user downloads:

Query attributes from database


Download file links from Azure

Generate Excel with attributes + attachment names/links

2️⃣ Database Design (PostgreSQL)


Use two tables.

Table 1 — risk_records

Stores the attributes.

Printed using ChatGPT to PDF, powered by PDFCrowd HTML to PDF API. 1/17
CREATE TABLE risk_records (
id SERIAL PRIMARY KEY,
risk_id VARCHAR(100),
attribute1 TEXT,
attribute2 TEXT,
created_at TIMESTAMP DEFAULT NOW()
);

Table 2 — attachments

Stores metadata about files.


CREATE TABLE attachments (
id SERIAL PRIMARY KEY,
risk_id VARCHAR(100),
file_name TEXT,
file_path TEXT,
uploaded_at TIMESTAMP DEFAULT NOW()
);

Example record:

id risk_id file_name file_path

1 RISK001 [Link] azure://container/risk001/[Link]

3️⃣ Upload API in FastAPI


Example:
from fastapi import FastAPI, UploadFile, File, Form
from typing import List

app = FastAPI()

@[Link]("/upload-risk/")
async def upload_risk(
risk_id: str = Form(...),
attribute1: str = Form(...),
attribute2: str = Form(...),
files: List[UploadFile] = File(...)
):

# 1 Save attributes to PostgreSQL


save_risk_record(risk_id, attribute1, attribute2)

uploaded_files = []

for file in files:


file_path = upload_to_azure(file, risk_id)

# Save file metadata


save_attachment(risk_id, [Link], file_path)

uploaded_files.append([Link])

return {"status": "success", "files": uploaded_files}

4️⃣ Upload File to Azure


Using Azure SDK.

Printed using ChatGPT to PDF, powered by PDFCrowd HTML to PDF API. 2/17
from [Link] import BlobServiceClient

connection_string = "AZURE_CONNECTION_STRING"

def upload_to_azure(file, risk_id):

blob_service = BlobServiceClient.from_connection_string(connection_string)

container = blob_service.get_container_client("risk-files")

blob_path = f"{risk_id}/{[Link]}"

blob = container.get_blob_client(blob_path)

blob.upload_blob([Link])

return blob_path

Storage structure:
risk-files/
RISK001/
[Link]
[Link]

5️⃣ Download Excel with Attachments


Use openpyxl or pandas.

Example:
import pandas as pd

def export_risk_data():

risks = get_risk_records()
attachments = get_attachments()

df_risk = [Link](risks)
df_files = [Link](attachments)

merged = df_risk.merge(df_files, on="risk_id", how="left")

file_path = "risk_export.xlsx"

merged.to_excel(file_path, index=False)

return file_path

Excel output:

Risk ID Attribute1 Attribute2 File Name

R001 High Security [Link]


R001 High Security [Link]

6️⃣ Optional: Add Download Links in Excel


You can store Azure SAS URLs.
Example:
download_link = generate_sas_url(blob_path)
Printed using ChatGPT to PDF, powered by PDFCrowd HTML to PDF API. 3/17
Excel column:

Attachment
[Link]

User can click to download.

7️⃣ Best Practices


✔ Store files in Azure not database
✔ Store file metadata in PostgreSQL
✔ Use risk_id as folder in storage
✔ Allow multiple attachments per risk
✔ Generate Excel with metadata + file links
8️⃣ Recommended Folder Structure
Backend project:
app/
[Link]
[Link]
[Link]
routes/
risk_routes.py
services/
azure_storage.py
excel_export.py

9️⃣ API Summary


API Purpose
POST /upload-risk upload attributes + attachments
GET /risk/{id} get risk details

GET /download-excel download Excel report


GET /download-file/{file_id} download specific attachment

🔟 Advanced (Enterprise Design)


If your system grows:

Add:

file size
content type

Printed using ChatGPT to PDF, powered by PDFCrowd HTML to PDF API. 4/17
versioning

virus scanning
audit logs

💡 Important Design Tip


Never store files inside PostgreSQL — always store them in Azure Blob Storage and only store the path
in the database.

✅ If you'd like, I can also show you a complete production-ready FastAPI code example (about 150
lines) that includes:

SQLAlchemy models
Azure upload

Multiple attachments
Excel export

It will save you a lot of development time.

is it possible to download files and attach to download excel as object

Yes, it is possible, but there are two different approaches, and one of them is usually better in real
systems.
I'll explain both clearly.

1️⃣ Option 1 — Embed Files Inside the Excel (as Objects)


You can download the files from Microsoft Azure storage and embed them inside the Excel workbook
as objects.
Excel allows files like PDF, Word, images to be embedded.

Python libraries:

XlsxWriter

openpyxl

The most reliable one for embedding objects is XlsxWriter.

Example

Printed using ChatGPT to PDF, powered by PDFCrowd HTML to PDF API. 5/17
import xlsxwriter

workbook = [Link]('risk_export.xlsx')
worksheet = workbook.add_worksheet()

[Link]('A1', 'Risk ID')


[Link]('B1', 'Attachment')

[Link]('A2', 'RISK001')

worksheet.insert_object('B2', '[Link]')

[Link]()

This will create:


risk_export.xlsx
└ embedded object: [Link]

When the user opens Excel, they will see an icon and can double-click to open the file.

But there are limitations ⚠️


If your users upload:

many files

large PDFs
large images

The Excel file can become extremely large and slow.

Example:

Attachments Excel Size


5 PDFs 10 MB

20 PDFs 80+ MB

This is why most enterprise systems do not embed files in Excel.

2️⃣ Option 2 — Put Download Links in Excel (Recommended)


Instead of embedding files, add download links.
Excel column example:

Risk ID Attribute Attachment

R001 High Download File

The link points to an Azure SAS URL.

Example:

Printed using ChatGPT to PDF, powered by PDFCrowd HTML to PDF API. 6/17
worksheet.write_url(
'C2',
'[Link]
string='Download Attachment'
)

User clicks → file downloads.

Why this approach is better

✔ Excel remains small


✔ Supports many attachments
✔ Works with large files
✔ Faster export
3️⃣ Enterprise Solution (Best Practice)
Most systems do this:

When user clicks Download:


Backend generates a ZIP package:
risk_export.zip
risk_export.xlsx
attachments/
RISK001_doc1.pdf
RISK001_doc2.pdf
RISK002_doc1.pdf

This avoids Excel limitations.

Python example:
import zipfile

with [Link]("[Link]", "w") as zipf:


[Link]("risk_export.xlsx")

for file in files:


[Link](file)

User downloads one ZIP containing everything.

4️⃣ Recommended Architecture for Your System


Since you're using:

FastAPI
PostgreSQL

Microsoft Azure storage

Best option:
✅ Export Excel with metadata
✅ Download attachments from Azure
Printed using ChatGPT to PDF, powered by PDFCrowd HTML to PDF API. 7/17
✅ Package everything in a ZIP
5️⃣ Visual Flow
User clicks "Download"

FastAPI:
1. Query attributes from PostgreSQL
2. Download attachments from Azure
3. Generate Excel
4. Put Excel + attachments into ZIP
5. Return ZIP

⭐ My Recommendation
Do NOT embed files in Excel.

Use:
Excel + attachments ZIP export

This is how banks, risk platforms, and audit tools handle document exports.

✅ If you'd like, I can also show a complete FastAPI implementation that:


downloads files from Azure

generates Excel
bundles everything into a ZIP

returns it via API

(about 120 lines of production-ready code).

can i get proper code with add field attributes, excel upload to file storage and path store in table
for respective risk ID, download the attributes in zip file format. End points - Add new Risk ID,
update existing Risk ID, download attributes

Below is a clean production-style example using:

FastAPI

PostgreSQL

Microsoft Azure Blob Storage

SQLAlchemy
Pandas

It supports exactly what you asked:

Printed using ChatGPT to PDF, powered by PDFCrowd HTML to PDF API. 8/17
✅ Add new Risk ID with attributes + attachments
✅ Update existing Risk ID
✅ Store files in Azure and path in DB
✅ Download Excel + attachments in ZIP
1️⃣ Install Dependencies
pip install fastapi uvicorn sqlalchemy psycopg2-binary pandas openpyxl azure-storage-blob python-multi

2️⃣ Project Structure


app/
├ [Link]
├ [Link]
├ [Link]
├ azure_storage.py
├ excel_service.py
└ [Link]

3️⃣ Database Connection


[Link]

from sqlalchemy import create_engine


from [Link] import sessionmaker, declarative_base

DATABASE_URL = "postgresql://user:password@localhost/riskdb"

engine = create_engine(DATABASE_URL)

SessionLocal = sessionmaker(autocommit=False, autoflush=False, bind=engine)

Base = declarative_base()

4️⃣ Database Models


[Link]

Printed using ChatGPT to PDF, powered by PDFCrowd HTML to PDF API. 9/17
from sqlalchemy import Column, Integer, String, ForeignKey
from [Link] import relationship
from database import Base

class Risk(Base):
__tablename__ = "risks"

id = Column(Integer, primary_key=True)
risk_id = Column(String, unique=True)
attribute1 = Column(String)
attribute2 = Column(String)

attachments = relationship("Attachment", back_populates="risk")

class Attachment(Base):
__tablename__ = "attachments"

id = Column(Integer, primary_key=True)
risk_id = Column(String, ForeignKey("risks.risk_id"))
file_name = Column(String)
file_path = Column(String)

risk = relationship("Risk", back_populates="attachments")

5️⃣ Azure Storage Upload


azure_storage.py

from [Link] import BlobServiceClient


import os

connection_string = "AZURE_CONNECTION_STRING"
container_name = "risk-files"

blob_service = BlobServiceClient.from_connection_string(connection_string)

def upload_file(file, risk_id):

blob_client = blob_service.get_blob_client(
container=container_name,
blob=f"{risk_id}/{[Link]}"
)

blob_client.upload_blob([Link], overwrite=True)

return f"{risk_id}/{[Link]}"

def download_file(blob_path, local_path):

blob_client = blob_service.get_blob_client(
container=container_name,
blob=blob_path
)

with open(local_path, "wb") as f:


data = blob_client.download_blob()
[Link]([Link]())

6️⃣ Excel Generation


excel_service.py

Printed using ChatGPT to PDF, powered by PDFCrowd HTML to PDF API. 10/17
import pandas as pd

def generate_excel(risks):

data = []

for risk in risks:


for file

Add new Risk ID with attributes


✅ Update existing Risk ID + attributes + attachments
✅ Store files in Azure and path in DB
✅ Download Excel + attachments in ZIP

Below is a complete working backend design using:

FastAPI

PostgreSQL

SQLAlchemy
Microsoft Azure Blob Storage

It implements exactly your requirements:

✅ Add new Risk ID + attributes + attachments


✅ Update existing Risk ID + attributes + attachments
✅ Store files in Azure & save path in DB
✅ Download Excel + attachments in ZIP
1️⃣ Install Required Libraries
pip install fastapi uvicorn sqlalchemy psycopg2-binary python-multipart pandas openpyxl azure-storage-

2️⃣ Project Structure


risk_service/

├── [Link]
├── [Link]
├── [Link]
├── azure_storage.py
├── excel_service.py
└── [Link]

3️⃣ Database Connection


[Link]

Printed using ChatGPT to PDF, powered by PDFCrowd HTML to PDF API. 11/17
from sqlalchemy import create_engine
from [Link] import sessionmaker, declarative_base

DATABASE_URL = "postgresql://user:password@localhost/riskdb"

engine = create_engine(DATABASE_URL)

SessionLocal = sessionmaker(bind=engine, autocommit=False, autoflush=False)

Base = declarative_base()

4️⃣ Database Tables


[Link]

from sqlalchemy import Column, Integer, String, ForeignKey


from [Link] import relationship
from database import Base

class Risk(Base):
__tablename__ = "risks"

id = Column(Integer, primary_key=True)
risk_id = Column(String, unique=True)
attribute1 = Column(String)
attribute2 = Column(String)

attachments = relationship("Attachment", back_populates="risk")

class Attachment(Base):
__tablename__ = "attachments"

id = Column(Integer, primary_key=True)
risk_id = Column(String, ForeignKey("risks.risk_id"))
file_name = Column(String)
file_path = Column(String)

risk = relationship("Risk", back_populates="attachments")

5️⃣ Azure File Upload Service


azure_storage.py

Printed using ChatGPT to PDF, powered by PDFCrowd HTML to PDF API. 12/17
from [Link] import BlobServiceClient

connection_string = "AZURE_CONNECTION_STRING"
container_name = "risk-files"

blob_service = BlobServiceClient.from_connection_string(connection_string)

def upload_file(file, risk_id):

blob_path = f"{risk_id}/{[Link]}"

blob_client = blob_service.get_blob_client(
container=container_name,
blob=blob_path
)

blob_client.upload_blob([Link], overwrite=True)

return blob_path

def download_file(blob_path, local_path):

blob_client = blob_service.get_blob_client(
container=container_name,
blob=blob_path
)

with open(local_path, "wb") as f:


data = blob_client.download_blob()
[Link]([Link]())

6️⃣ Excel Export Service


excel_service.py

import pandas as pd

def generate_excel(risks):

rows = []

for risk in risks:


for file in [Link]:

[Link]({
"Risk ID": risk.risk_id,
"Attribute1": risk.attribute1,
"Attribute2": risk.attribute2,
"Attachment": file.file_name
})

df = [Link](rows)

file_path = "risk_export.xlsx"

df.to_excel(file_path, index=False)

return file_path

7️⃣ API Endpoints


[Link]

Printed using ChatGPT to PDF, powered by PDFCrowd HTML to PDF API. 13/17
from fastapi import APIRouter, UploadFile, File, Form, Depends
from [Link] import Session
from database import SessionLocal
from models import Risk, Attachment
from azure_storage import upload_file, download_file
from excel_service import generate_excel
import os
import zipfile

router = APIRouter()

def get_db():
db = SessionLocal()
try:
yield db
finally:
[Link]()

# ----------------------------
# Add New Risk
# ----------------------------
@[Link]("/risk")
async def add_risk(
risk_id: str = Form(...),
attribute1: str = Form(...),
attribute2: str = Form(...),
files: list[UploadFile] = File(None),
db: Session = Depends(get_db)
):

risk = Risk(
risk_id=risk_id,
attribute1=attribute1,
attribute2=attribute2
)

[Link](risk)
[Link]()

if files:
for file in files:

blob_path = upload_file(file, risk_id)

attachment = Attachment(
risk_id=risk_id,
file_name=[Link],
file_path=blob_path
)

[Link](attachment)

[Link]()

return {"message": "Risk created"}

8️⃣ Update Existing Risk

Printed using ChatGPT to PDF, powered by PDFCrowd HTML to PDF API. 14/17
@[Link]("/risk/{risk_id}")
async def update_risk(
risk_id: str,
attribute1: str = Form(None),
attribute2: str = Form(None),
files: list[UploadFile] = File(None),
db: Session = Depends(get_db)
):

risk = [Link](Risk).filter(Risk.risk_id == risk_id).first()

if not risk:
return {"error": "Risk not found"}

if attribute1:
risk.attribute1 = attribute1

if attribute2:
risk.attribute2 = attribute2

if files:
for file in files:

blob_path = upload_file(file, risk_id)

attachment = Attachment(
risk_id=risk_id,
file_name=[Link],
file_path=blob_path
)

[Link](attachment)

[Link]()

return {"message": "Risk updated"}

9️⃣ Download Attributes + Attachments


@[Link]("/download")
def download_all(db: Session = Depends(get_db)):

risks = [Link](Risk).all()

excel_path = generate_excel(risks)

[Link]("downloads", exist_ok=True)

for risk in risks:


for file in [Link]:

local_path = f"downloads/{file.file_name}"

download_file(file.file_path, local_path)

zip_name = "risk_export.zip"

with [Link](zip_name, "w") as zipf:

[Link](excel_path)

for file in [Link]("downloads"):


[Link](f"downloads/{file}", f"attachments/{file}")

return {"zip_file": zip_name}

🔟 Main Application
Printed using ChatGPT to PDF, powered by PDFCrowd HTML to PDF API. 15/17
[Link]

from fastapi import FastAPI


from routes import router
from database import Base, engine

app = FastAPI()

[Link].create_all(bind=engine)

app.include_router(router)

Run server:
uvicorn main:app --reload

API Summary
Endpoint Method Purpose

/risk POST Add new risk + attributes + attachments


/risk/{risk_id} PUT Update risk + add attachments
/download GET Download Excel + attachments ZIP

Example Export Structure


When user downloads:
risk_export.zip

├── risk_export.xlsx

└── attachments/
[Link]
audit_report.docx

Excel contains:

Risk ID Attribute1 Attribute2 Attachment


R001 High Security [Link]

⭐ Enterprise Improvements (Recommended)


You can enhance this by adding:
✔ file size
✔ file type validation
✔ virus scanning
✔ versioning of attachments
✔ signed download URLs
Printed using ChatGPT to PDF, powered by PDFCrowd HTML to PDF API. 16/17
✅ If you want, I can also show you a much better enterprise architecture used in banks and risk
platforms where:

uploads are 10x faster

downloads stream directly from Azure


supports thousands of attachments

This would be useful if your system will scale.

Printed using ChatGPT to PDF, powered by PDFCrowd HTML to PDF API. 17/17

You might also like