System Architecture
DOPE is built on a modular architecture designed for low-latency data processing and real-time pattern detection. This document describes the system components and their interactions.
High-Level Architecture
DOPE follows a producer-consumer pattern with multiple specialized components working together to process market data and detect patterns.
Backend (Python)
Data collection, processing, and WebSocket server. Handles all real-time computation and pattern detection.
Frontend (Next.js)
React-based UI that connects to the backend WebSocket for real-time data visualization and user interaction.
Data Collection Layer
Each exchange is connected via a dedicated venue connector that handles WebSocket connections, message parsing, and data normalization.
Venue Connectors
Platform-specific implementations for each exchange:
- Binance: Spot and USDⓈ-M perpetuals via combined stream API
- Bybit: V5 API with order book snapshots and incremental updates
- OKX: V5 WebSocket with books and trades channels
- Hyperliquid: DEX API with l2Book and trades subscriptions
Connection Management
All connectors implement automatic reconnection with exponential backoff. Health monitoring tracks connection status, subscription confirmations, and error rates. Failed connections are retried with increasing delays (1s, 2s, 4s, up to 60s max).
Processing Pipeline
The processing pipeline consists of multiple asynchronous consumers that process different types of market data events.
Trade Consumer
Processes trade events to calculate volume metrics and detect trading bursts.
- Maintains rolling baselines for buy/sell volume
- Calculates Z-scores for volume bursts
- Buckets trades into time windows (default: 50ms)
Book Consumer
Processes order book updates to maintain current state and calculate depth metrics.
- Maintains in-memory order book state per venue
- Handles both snapshots and incremental updates
- Calculates depth within configurable basis point windows
Sampler Loop
Periodic sampling (default: 10ms) that reads current order book state and generates feature events for broadcasting.
- Calculates mid prices and deltas
- Reads depth metrics from order books
- Packages data into feature events
- Broadcasts to all connected WebSocket clients
Pattern Detection System
DOPE includes specialized modules for detecting different types of market patterns.
Jump Detection
Monitors price movements and flags significant jumps based on configurable thresholds. Uses a rolling window comparison to detect relative price changes.
Lead-Lag Linker
Matches jumps across venues to identify lead-lag patterns. Validates relationships based on time windows, direction, and magnitude ratios.
WebSocket Broadcasting
The WebSocket server uses aiohttp to handle both HTTP health checks and WebSocket connections on the same port, enabling deployment on platforms like Render that require health check endpoints.
Features
- Supports multiple concurrent client connections
- Automatic cleanup of disconnected clients
- Health check endpoints for deployment platforms
- Efficient message broadcasting to all clients
- Compatible with both websockets library and aiohttp WebSocket clients
Deployment Architecture
DOPE is designed to run on standard cloud platforms with minimal configuration.
Backend (Render)
Python service running the data collection and processing engine.
- • WebSocket server on configurable port
- • Health check endpoints
- • Automatic scaling support
Frontend (Vercel)
Next.js application with server-side rendering and static generation.
- • API routes for external data
- • Optimized static assets
- • Edge network distribution