Part-1: Codebase Overview, File Roles,
Database & Execution Guide
Project Overview
This project is a fully automated backend system for extracting, monitoring, and
broadcasting trading signals from TradingView's Strategy Tester. It includes robust scraping
logic, an event-driven FastAPI backend, persistent database logging, and automated
end-to-end testing. The system is designed for resilience, idempotency, and can easily be
integrated with an executor/frontend for automated trading or notifications.
File Structure & Roles
1. [Link]
● Purpose: The main FastAPI web server for managing chart signal monitoring, event
streaming (SSE), and webhook notifications.
● Key Functions:
○ Exposes endpoints to start/stop monitoring (/charts), check status, stream
events, and manage the webhook event buffer (/debug/executor).
○ Launches a worker thread (ChartWorker) for each unique chart/identifier.
○ Handles SSE subscriptions for real-time event feeds to clients/executors.
○ Ensures at-most-once delivery for each trade open/close notification.
○ Manages lifecycle, logging, and event publishing (open, close, alive, dead).
● Typical Usage: Run with uvicorn api:app --reload. Interacts with both the
browser scraper and the database.
2. chart_worker.py
● Purpose: Implements ChartWorker, the background worker class responsible for
all chart scraping and event generation.
● Key Functions:
○ Starts SeleniumBase browser, handles login, cookies, and navigation.
○ Uses core scraping utilities from [Link] to track the TradingView
trades table.
○ Publishes 'open' (entry), 'close' (exit), and 'alive/dead' events back to [Link].
○ Handles recovery, refreshes, and guarantees idempotency (no duplicate
event emission on refresh).
○ Writes each unique trade event to the database via [Link].
3. [Link]
● Purpose: All logic for interacting with TradingView via SeleniumBase (browser
automation), including login, table scraping, cookie management, and CAPTCHA
solving.
● Key Functions:
○ Exposes robust scraping functions for extracting and parsing trade events
(open/close).
○ Maintains browser login/session, solves CAPTCHA via CapSolver.
○ Exports utilities for single-pass (catch-up) and live event-driven scraping.
○ Used exclusively by chart_worker.py—never called directly by the API
layer.
4. [Link]
● Purpose: Database adapter for persisting trade events.
● Key Functions:
○ Exposes functions to connect to the database, insert or update (upsert)
trades, and retrieve records.
○ Used by chart_worker.py to log all trade open/close events and state.
5. test_system.py
● Purpose: Full automated end-to-end test driver for the backend API, event feeds,
and webhook system.
● Key Functions:
○ Runs a complete workflow: launches chart monitoring, listens for SSE events,
validates payloads, checks webhook event buffer, and triggers a clean
shutdown.
○ Validates at-most-once event emission and full signal integrity.
○ Prints step-by-step results, highlighting any missing/duplicate/invalid events.
● Typical Usage: Run after starting the server to confirm full system correctness.
6. [Link]
● Purpose: All required dependencies (SeleniumBase, FastAPI, httpx, dotenv, etc.).
● Usage: pip install -r [Link] in a fresh virtual environment.
7. create_trades.sql
● Purpose: Schema definition for the trades database table (for SQLite or Postgres).
● Usage: Should be executed to create the trades table before starting the backend.
How to Set Up and Run Everything
1. Python Environment & Dependencies
Create and activate a virtual environment:
python -m venv .venv
# On Windows:
.venv\Scripts\activate
# On Linux/Mac:
source .venv/bin/activate
●
Install requirements:
pip install -r [Link]
●
2. Database Setup
● By default, the backend uses SQLite (or Postgres, if configured in [Link]).
To set up the schema:
# Open SQLite CLI, or connect to your Postgres database as needed:
sqlite3 [Link]
# Then run:
.read create_trades.sql
# Or copy-paste the SQL in create_trades.sql:
-- Example (SQLite CLI):
CREATE TABLE IF NOT EXISTS trades (
id TEXT PRIMARY KEY,
entry_type TEXT,
entry_signal TEXT,
entry_price REAL,
entry_time TEXT,
exit_price REAL,
exit_time TEXT,
exit_signal TEXT,
raw_json TEXT
);
●
● Note: If using Postgres, create the database and run the SQL there. Set connection
details in [Link] as needed.
3. Environment Variables
Copy your TradingView login credentials and CapSolver API key to a .env file in the project
root:
EMAIL=your_tradingview_email
PASSWORD=your_tradingview_password
CAPSOLVER_API_KEY=your_capsolver_key
●
4. Running the Server
Start the FastAPI server (after DB is initialized):
uvicorn api:app --reload
●
5. Running End-to-End Tests
In another terminal (with the venv activated):
python test_system.py
● This launches the full test script, which starts monitoring, listens to all events,
validates their format and ordering, and prints a pass/fail summary.
How to Validate Results
● Watch the terminal/logs:
○ [Link] logs every chart worker, event emission, webhook, and SSE event.
○ test_system.py prints all received events and validates:
■ Only one open/close per trade.
■ No duplicates.
■ All keys present.
■ Counts of open/close/alive/dead as expected.
● Database:
○ Inspect the trades table after a run. Each row should represent a trade with
open/close data (use sqlite3 [Link] or your SQL client).
● Webhooks:
○ All event payloads are also sent to the /debug/executor endpoint for audit.
● Any errors, duplicates, or missing events will be flagged in the logs and test script
output.