0% found this document useful (0 votes)
1 views13 pages

Progres Roadmap

The document outlines a detailed 5-day sprint plan for the LUMORA project, focusing on the implementation of an AI-driven alert correlation system. It includes phases for foundation, AI core, intelligence layer, backend API, frontend development, hardening, and demo preparation, with specific goals, deliverables, and testing strategies for each phase. Additionally, it highlights risks and mitigation strategies, emphasizing the critical path for project completion.

Uploaded by

Soumya Sawant
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
1 views13 pages

Progres Roadmap

The document outlines a detailed 5-day sprint plan for the LUMORA project, focusing on the implementation of an AI-driven alert correlation system. It includes phases for foundation, AI core, intelligence layer, backend API, frontend development, hardening, and demo preparation, with specific goals, deliverables, and testing strategies for each phase. Additionally, it highlights risks and mitigation strategies, emphasizing the critical path for project completion.

Uploaded by

Soumya Sawant
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

LUMORA — Implementation Roadmap

(Deliverable 5)

5-Day Sprint Plan — Synergy 2026 HPE Hackathon

Overview

Day Focus Hours Status

Day 1 (Jul 13) Foundation + AI Core + Intelligence 12h ✅ COMPLETE


Day 2 (Jul 14) Backend API + Frontend + Integration 12h ✅ COMPLETE
Day 3 (Jul 15) Auth + Persistence + WebSocket + Frontend polish 12h ✅ COMPLETE
Day 4 (Jul 16-17) Hardening + Tuning + Edge Cases 12h ⚠️ IN PROGRESS
Day 5 (Jul 17-18) Demo Prep + Rehearsal + Polish 8h 📋 NEXT

Phase 1 — Foundation (Day 1, First 3 Hours) ✅


COMPLETE

Goal

Establish project skeleton, data models, and data ingestion pipeline.

Deliverables

Project directory structure with all package __init__.py files


Pydantic Alert model with BGL topology auto-parser
Pydantic Incident model with CorrelationResult container
BGL log regex parser with 9-field extraction
BGL dataset auto-downloader from Loghub GitHub
Configuration file ([Link])
Requirements file ([Link])
Virtual environment setup

Files to Implement

File Lines Purpose

models/[Link] ~90 Alert schema with BGL topology parser, severity normalization

models/incident.p Incident, IncidentReport, CorrelationEvidence,


y ~80
CorrelationResult

models/__init__.p
y ~5 Module exports

data/bgl_parser.p
y ~130 Regex parser, ground-truth inference, file loader

data/downloader.p
y ~45 Auto-download BGL_2k.log from Loghub

data/__init__.py ~10 Module exports

[Link] ~35 All parameters, weights, API key placeholders

[Link] ~18 Pinned dependency versions

Dependencies

Python 3.10+
pydantic >= 2.9.0
pyyaml >= 6.0.2

Testing

python -c "from [Link] import Alert; print('Alert OK')"


python -c "from [Link] import Incident, CorrelationResult; print
python -c "from data.bgl_parser import load_bgl_file; alerts=load_bgl_fil

Expected Output

Alert model auto-parses R04-M1-N4-I:J18-U11 into {rack: "R04", midplane:


"M1", node_card: "N4"}
BGL parser extracts 143 alerts from BGL_2k.log (alert_only=True)

Risk
BGL log format edge cases (non-standard lines, truncated entries)
Mitigation: Regex with try/except, skip unparseable lines

Priority: 🔴 P0
Phase 2 — AI Core Pipeline (Day 1, Next 6 Hours) ✅
COMPLETE

Goal

Implement the complete 7-step correlation pipeline (dedup → embed → score → fuse → cluster → root
cause → orchestrator).

Deliverables

SHA-256 deduplication module


Sentence-transformer embedding wrapper with fallback
Three-signal scorer (temporal, semantic, topological)
Weighted fusion matrix builder
HDBSCAN clustering wrapper with fallback
Composite root cause heuristic
AlertCorrelationEngine orchestrator class

Files to Implement

File Lines Purpose

core/[Link] ~41 SHA-256 hash on normalized fingerprint

core/[Link] ~36 Sentence-transformer batch encoding + token-hash fallback

core/[Link] ~52 Gaussian temporal + cosine semantic + hierarchical topological

core/[Link] ~66 Weighted linear combination → N×N distance matrix

core/[Link] ~62 HDBSCAN + connected-component fallback

core/root_cause.py ~49 Composite heuristic (time + severity + centrality)

core/[Link] ~200 AlertCorrelationEngine class + run() method

core/__init__.py ~20 Module exports


Dependencies

sentence-transformers >= 3.0.0


scikit-learn >= 1.6.0
numpy >= 1.26.0

Testing

python -c "
from data.bgl_parser import load_bgl_file
from [Link] import AlertCorrelationEngine
alerts = load_bgl_file('data/sample_bgl.log')
engine = AlertCorrelationEngine()
result = [Link](alerts)
print(f'{len([Link])} incidents from {len(alerts)} alerts')
"

Expected Output

14 sample alerts → 3-5 incidents


No crashes with empty input, single alert, all-noise scenarios

Risk

Sentence-transformer model download on first run (~80MB)


HDBSCAN may produce all-noise results with default parameters
Mitigation: Token-hash fallback for embeddings; connected-component fallback for clustering

Priority: 🔴 P0
Phase 3 — Intelligence Layer (Day 1-2, 3 Hours) ✅
COMPLETE

Goal

Add LLM summarization, evaluation metrics, weight optimization, explainability, and vision analysis.
Deliverables

3-tier LLM summarizer (Gemini → Groq → Template) with structured reports


Evaluation metrics (NMI, ARI, Silhouette) with pure-Python fallbacks
Grid search weight optimizer (coarse + fine-grained)
Cross-cluster explainability module
Vision analyzer (Gemini Vision screenshot analysis)

Files to Implement

File Lines Purpose

3-tier LLM chain, structured prompts, deterministic


core/[Link] ~292
builders

core/[Link] ~147 NMI, ARI, Silhouette with sklearn + pure-numpy fallbacks

Coarse (4 candidates) + fine-grained (66 candidates) grid


core/[Link] ~125
search

core/explainability.p
y ~84 Cross-cluster links, human-readable cluster explanations

core/vision_analyzer. Gemini Vision screenshot→text extraction→incident


py ~121
correlation

Dependencies

google-generativeai >= 0.8.6


groq >= 0.9.0

Testing

# Test template fallback (no API keys)


python -c "
from [Link] import Summarizer
from [Link] import Alert
from datetime import datetime
s = Summarizer() # no keys
alert = Alert(id='1', timestamp=[Link](), severity='critical', serv
report = s.generate_report([alert], alert)
print(f'Generated by: {report.generated_by}')
print(f'Summary: {report.executive_summary[:100]}')
"
Expected Output

Template fallback always produces structured report


NMI/ARI/Silhouette computed without errors
Optimizer returns best weights with metrics

Risk

Gemini/Groq API rate limits during demo


Mitigation: Template fallback + pre-cached summaries for demo scenarios

Priority: 🔴 P0 (summarizer, evaluator), 🟡 P1 (optimizer, vision)


Phase 4 — Backend API (Day 2, 4 Hours) ✅ COMPLETE
Goal

Expose the AI pipeline via a RESTful API with authentication, rate limiting, and persistence.

Deliverables

FastAPI entry point with startup hook


15+ REST API endpoints
WebSocket pipeline streaming endpoint
SQLite persistence (3-table schema)
JWT authentication (Supabase)
Rate limiting middleware
Request logging middleware
Global exception handler
CORS middleware configuration

Files to Implement

File Lines Purpose

[Link] ~247 FastAPI app, startup, auth routes, WS endpoint

api/[Link] ~385 15+ REST endpoints with validation

services/[Link] ~361 SQLite CRUD (runs, alerts, incidents)


File Lines Purpose

auth/supabase_client.py ~85 Supabase sign up/sign in

auth/jwt_handler.py ~50 JWT creation/validation

auth/[Link] ~90 FastAPI auth dependency injection

auth/[Link] ~25 Auth request/response schemas

middleware/rate_limiter.py ~150 Token bucket rate limiting

logging_config.py ~100 Structured JSON logging

ws/[Link] ~100 WebSocket manager + pipeline events

Dependencies

fastapi >= 0.115.0


uvicorn >= 0.30.0
httpx >= 0.27.0
python-multipart >= 0.0.9
PyJWT >= 2.8.0
python-dotenv >= 1.0.0

Testing

# Start server
cd alert_engine && python [Link] &

# Test health
curl [Link]

# Test status
curl [Link]

# Test dataset load


curl -X POST [Link]

# Test incidents
curl [Link]

Expected Output
Server starts on port 8000
Swagger docs at /docs
Auto-loads BGL_2k.log on startup
All endpoints return valid JSON

Risk

FastAPI import circular dependencies (main ↔ routes)


Mitigation: Lazy imports inside route handlers

Priority: 🔴 P0
Phase 5 — Frontend (Day 2-3, 8 Hours) ✅ COMPLETE
Goal

Build a production-quality React SPA with 8 pages, WebSocket integration, and polished UI.

Deliverables

React project setup (Vite + TailwindCSS + TypeScript)


App layout with sidebar navigation
Dashboard page (KPIs, metrics)
Live Alerts page (filterable table)
Incidents page (expandable cards)
Analytics page (clustering metrics)
Weight Optimization page (sliders + optimizer)
Screenshot Analysis page (upload + analysis)
History page (past runs)
Settings page (configuration)
Login/Signup pages (Supabase auth)
API client service
Auth context provider

Files to Implement

Directory Files Purpose

src/pages/ 10 page components All route pages


Directory Files Purpose

src/components/ 8 component directories Reusable UI elements

src/services/ API client HTTP + WebSocket clients

src/contexts/ Auth context Authentication state

src/hooks/ Custom hooks Data fetching hooks

src/config/ Routes config Route path constants

src/types/ TypeScript types Alert, Incident, etc.

src/animations/ Motion presets Framer Motion variants

src/lib/ Utilities cn(), formatters

Dependencies

react 19, react-dom 19, react-router-dom 7


@tanstack/react-query 5
tailwindcss 4, framer-motion 12
recharts 3.9, lucide-react 1
@supabase/supabase-js 2
axios 1, zod 4

Testing

cd frontend
npm run build # Verify no TypeScript errors
npm run dev # Manual UI testing

Expected Output

8 pages render without errors


API integration works (fetches incidents, alerts)
Auth flow works (login → protected routes)
Build produces dist/ folder served by FastAPI

Risk

TypeScript type mismatches with backend API


Mitigation: Zod runtime validation, React Query error boundaries
Priority: 🔴 P0 (Dashboard, Incidents), 🟡 P1 (others)
Phase 6 — Hardening & Tuning (Day 4, 8 Hours) ⚠️ IN
PROGRESS

Goal

Make the system demo-reliable. Tune parameters. Handle edge cases. Test with real API keys.

Deliverables

End-to-end integration test on full BGL_2k.log


HDBSCAN parameter tuning (min_cluster_size, min_samples)
Fusion weight optimization on real data
LLM API integration test (Gemini + Groq)
Template fallback test (remove keys)
Edge case handling (empty, single alert, all-noise)
LLM timeout wrappers
Pre-built demo scenarios

Files to Modify/Create

File Action Purpose

[Link] MODIFY Update with optimized weights + HDBSCAN params

core/[Link] MODIFY Add timeout wrappers for LLM calls

core/[Link] VERIFY Test all edge case paths

tests/test_integration.py CREATE Full pipeline integration test

data/scenarios/ CREATE Pre-built demo scenario JSON files

Dependencies

All previous phases complete


Gemini API key (from [Link])
Groq API key (from [Link])
Testing

# Full pipeline test


cd alert_engine
pytest tests/ -v

# Manual integration test


python -c "
from data.bgl_parser import load_bgl_file
from [Link] import AlertCorrelationEngine
import yaml, time

with open('[Link]') as f:
config = yaml.safe_load(f)
alerts = load_bgl_file('data/BGL_2k.log', max_lines=2000, alert_only=True
engine = AlertCorrelationEngine(config=config)
start = [Link]()
result = [Link](alerts)
elapsed = [Link]() - start
print(f'Alerts: {len(alerts)}')
print(f'Incidents: {len([Link])}')
print(f'Time: {elapsed:.1f}s')
print(f'NMI: {[Link](\"nmi\", \"N/A\")}')
print(f'Silhouette: {[Link](\"silhouette\", \"N/A\")}')
"

Expected Output

Processing < 15s for 143 alerts


NMI > 0.5 (ideally > 0.7 after tuning)
No crashes with any input
LLM fallback chain tested

Risk

HDBSCAN produces too many noise points (under-clustering)


HDBSCAN produces one giant cluster (over-clustering)
Mitigation: Systematic grid search over min_cluster_size ∈ {2,3,4,5} × min_samples ∈ {1,2,3}

Priority: 🔴 P0
Phase 7 — Demo Preparation (Day 5, 6 Hours) 📋 NEXT
Goal

Create a polished, rehearsed demo that maximizes hackathon evaluation score.

Deliverables

4-minute demo script written


3 demo scenarios pre-loaded (different failure types)
Pre-cached LLM summaries for demo scenarios
3 complete demo rehearsals
Backup video recorded
Judge Q&A answers prepared (20 questions)
PowerPoint presentation (8 slides)
Final README polish

Files to Create

File Purpose

DEMO_SCRIPT.md 4-minute presentation script

data/scenarios/scenario_1.json Pre-built demo scenario

data/scenarios/scenario_2.json Pre-built demo scenario

data/scenarios/scenario_3.json Pre-built demo scenario

Testing

Full demo run-through 3x


Backup video works independently
All API endpoints respond during demo

Expected Output

4-minute demo runs smoothly


Handles judge interruptions gracefully
Backup video available if live demo fails
20 Q&A answers rehearsed
Risk

Live demo failure (API crash, network issue)


Mitigation: Backup video, pre-cached LLM responses, template fallback

Priority: 🔴 P0
Critical Path Analysis
Phase 1 (Foundation) ──┐

Phase 2 (AI Core) ────┼──▶ Phase 4 (Backend) ──▶ Phase 6 (Hardening) ──▶
Phase 7 (Demo)

Phase 3 (Intelligence) ┘
Phase 5 (Frontend) ──────────────────────┘

Critical Path: Foundation → AI Core → Backend → Hardening → Demo

Frontend is parallel to backend and can lag by ~1 day without blocking.

Risk Mitigation Matrix

Risk Probability Impact Mitigation

HDBSCAN all-noise Tune min_cluster_size/min_samples; connected-


Medium High
output component fallback

LLM API rate limit Pre-cache summaries; Groq backup; template


Medium Medium
during demo fallback

Model download fails at Pre-download model in .venv/; include in Docker


Low High
demo image

Frontend build breaks Low Medium Pin all npm dependencies; test build before demo

Backup video; global exception handler; rehearse


Demo crashes Low Critical
3x

NMI score too low Medium Medium Grid search optimizer; manual weight tuning

SQLite corruption Very Low Medium WAL mode; backup DB file before demo

You might also like