Methodology & Algorithms
This document describes the mathematical models, algorithms, and statistical methods used in DOPE for pattern detection and market analysis.
Jump Detection Algorithm
Jump detection identifies sudden price movements that exceed configurable thresholds. The algorithm uses a rolling window comparison to detect relative price changes.
Algorithm
- Maintain a rolling time series of mid prices (default window: 10 seconds)
- Compare current mid price to the minimum/maximum price in the rolling window
- Calculate basis point change:
bps = (current - reference) / reference × 10000 - If |bps| exceeds threshold (default: 5 bps), flag as a jump
- Apply minimum separation time (default: 100ms) to prevent duplicate detections
Configuration Parameters
- window_ms: Rolling window size in milliseconds (default: 1500ms)
- threshold_bps: Minimum basis point change to trigger jump (default: 5.0)
- threshold_bps_min: Minimum threshold that decays over time (default: 2.5)
- threshold_decay_s: Time in seconds for threshold to decay (default: 300s)
- min_separation_ms: Minimum time between jump detections (default: 100ms)
Learn more about lead-lag patternsand how jumps are used to detect cross-exchange relationships.
Lead-Lag Pattern Matching
The linker module matches jumps across venues to identify lead-lag relationships. This uses a time-windowed matching algorithm with magnitude validation.
Matching Algorithm
- When a jump is detected on venue A (leader), enter follow window mode
- Monitor all other venues for jumps within the follow window (default: 2000ms)
- For each candidate jump on venue B (follower), validate:
- Same direction (both positive or both negative)
- Magnitude ratio: 0.5 ≤ |follower_bps| / |leader_bps| ≤ 1.5
- Time difference within follow window
- If validated, create a Link event with lag time and magnitude data
Order Book Depth Calculations
Depth metrics are calculated by summing order book volume within specific price ranges, measured in basis points from the best bid/ask.
Depth Within Basis Points
For a given side (bid or ask) and basis point window:
- Get best bid/ask price (B)
- Calculate price range: [B, B × (1 ± bps/10000)]
- Sum all order sizes within this range
- Return total depth
Imbalance Calculation
Imbalance measures the ratio of bid to ask volume within a basis point window:
Values range from 0.0 (all asks) to 1.0 (all bids), with 0.5 indicating balance.
Read more about order book depth analysisand how to interpret these metrics.
Volume Burst Z-Score
Z-scores measure how many standard deviations the current trading volume is from the baseline average. This helps identify statistically significant volume spikes.
Calculation
- Maintain rolling statistics (mean and standard deviation) over baseline window (default: 900s)
- Bucket trades into time windows (default: 50ms) and sum volume per bucket
- For each bucket, calculate:
z = (volume - mean) / std_dev - Separate calculations for buy and sell volume
Data Structures
Efficient data structures are critical for real-time processing performance.
Order Book
Uses SortedDict (from sortedcontainers) for O(log n) insertions and O(1) best price access.
- • Bids: Sorted ascending (best bid is last item)
- • Asks: Sorted ascending (best ask is first item)
- • Automatic trimming to max_levels (default: 50)
- • Supports snapshot and incremental updates
Time Series
Rolling time series for price history with automatic expiration of old data points.
- • Fixed horizon (default: 10 seconds)
- • Automatic cleanup of expired entries
- • Efficient min/max queries for jump detection
Rolling Statistics
Maintains mean and standard deviation over a rolling time window for Z-score calculations.
- • Welford's online algorithm for variance
- • Automatic expiration of old data points
- • O(1) updates and queries