popov/gogo2
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
fa07265a16ab998cb3a7323c64f53b595a79ef0f
gogo2/.kiro/specs/multi-modal-trading-system/design.md
Dobromir Popov fa07265a16 wip training
2025-07-24 15:27:32 +03:00

701 lines
26 KiB
Markdown
Raw Blame History

# Multi-Modal Trading System Design Document
## Overview
The Multi-Modal Trading System is designed as an advanced algorithmic trading platform that combines Convolutional Neural Networks (CNN) and Reinforcement Learning (RL) models orchestrated by a decision-making module. The system processes multi-timeframe and multi-symbol market data (primarily ETH and BTC) to generate trading actions.
This design document outlines the architecture, components, data flow, and implementation details for the system based on the requirements and existing codebase.
## Architecture
The system follows a modular architecture with clear separation of concerns:
```mermaid
graph TD
A[Data Provider] --> B[Data Processor] (calculates pivot points)
B --> C[CNN Model]
B --> D[RL(DQN) Model]
C --> E[Orchestrator]
D --> E
E --> F[Trading Executor]
E --> G[Dashboard]
F --> G
H[Risk Manager] --> F
H --> G
```
### Key Components
1. **Data Provider**: Centralized component responsible for collecting, processing, and distributing market data from multiple sources.
2. **Data Processor**: Processes raw market data, calculates technical indicators, and identifies pivot points.
3. **CNN Model**: Analyzes patterns in market data and predicts pivot points across multiple timeframes.
4. **RL Model**: Learns optimal trading strategies based on market data and CNN predictions.
5. **Orchestrator**: Makes final trading decisions based on inputs from both CNN and RL models.
6. **Trading Executor**: Executes trading actions through brokerage APIs.
7. **Risk Manager**: Implements risk management features like stop-loss and position sizing.
8. **Dashboard**: Provides a user interface for monitoring and controlling the system.
## Components and Interfaces
### 1. Data Provider
The Data Provider is the foundation of the system, responsible for collecting, processing, and distributing market data to all other components.
#### Key Classes and Interfaces
- **DataProvider**: Central class that manages data collection, processing, and distribution.
- **MarketTick**: Data structure for standardized market tick data.
- **DataSubscriber**: Interface for components that subscribe to market data.
- **PivotBounds**: Data structure for pivot-based normalization bounds.
#### Implementation Details
The DataProvider class will:
- Collect data from multiple sources (Binance, MEXC)
- Support multiple timeframes (1s, 1m, 1h, 1d)
- Support multiple symbols (ETH, BTC)
- Calculate technical indicators
- Identify pivot points
- Normalize data
- Distribute data to subscribers
- Calculate any other algoritmic manipulations/calculations on the data
- Cache up to 3x the model inputs (300 ticks OHLCV, etc) data so we can do a proper backtesting in up to 2x time in the future
Based on the existing implementation in `core/data_provider.py`, we'll enhance it to:
- Improve pivot point calculation using reccursive Williams Market Structure
- Optimize data caching for better performance
- Enhance real-time data streaming
- Implement better error handling and fallback mechanisms
### BASE FOR ALL MODELS ###
- ***INPUTS***: COB+OHCLV data frame as described:
- OHCLV: 300 frames of (1s, 1m, 1h, 1d) ETH + 300s of 1s BTC
- COB: for each 1s OHCLV we have +- 20 buckets of COB ammounts in USD
- 1,5,15 and 60s MA of the COB imbalance counting +- 5 COB buckets
- ***OUTPUTS***: suggested trade action (BUY/SELL)
### 2. CNN Model
The CNN Model is responsible for analyzing patterns in market data and predicting pivot points across multiple timeframes.
#### Key Classes and Interfaces
- **CNNModel**: Main class for the CNN model.
- **PivotPointPredictor**: Interface for predicting pivot points.
- **CNNTrainer**: Class for training the CNN model.
- ***INPUTS***: COB+OHCLV+Old Pivots (5 levels of pivots)
- ***OUTPUTS***: next pivot point for each level as price-time vector. (can be plotted as trend line) + suggested trade action (BUY/SELL)
#### Implementation Details
The CNN Model will:
- Accept multi-timeframe and multi-symbol data as input
- Output predicted pivot points for each timeframe (1s, 1m, 1h, 1d)
- Provide confidence scores for each prediction
- Make hidden layer states available for the RL model
Architecture:
- Input layer: Multi-channel input for different timeframes and symbols
- Convolutional layers: Extract patterns from time series data
- LSTM/GRU layers: Capture temporal dependencies
- Attention mechanism: Focus on relevant parts of the input
- Output layer: Predict pivot points and confidence scores
Training:
- Use programmatically calculated pivot points as ground truth
- Train on historical data
- Update model when new pivot points are detected
- Use backpropagation to optimize weights
### 3. RL Model
The RL Model is responsible for learning optimal trading strategies based on market data and CNN predictions.
#### Key Classes and Interfaces
- **RLModel**: Main class for the RL model.
- **TradingActionGenerator**: Interface for generating trading actions.
- **RLTrainer**: Class for training the RL model.
#### Implementation Details
The RL Model will:
- Accept market data, CNN model predictions (output), and CNN hidden layer states as input
- Output trading action recommendations (buy/sell)
- Provide confidence scores for each action
- Learn from past experiences to adapt to the current market environment
Architecture:
- State representation: Market data, CNN model predictions (output), CNN hidden layer states
- Action space: Buy, Sell
- Reward function: PnL, risk-adjusted returns
- Policy network: Deep neural network
- Value network: Estimate expected returns
Training:
- Use reinforcement learning algorithms (DQN, PPO, A3C)
- Train on historical data
- Update model based on trading outcomes
- Use experience replay to improve sample efficiency
### 4. Orchestrator
The Orchestrator serves as the central coordination hub of the multi-modal trading system, responsible for data subscription management, model inference coordination, output storage, training pipeline orchestration, and inference-training feedback loop management.
#### Key Classes and Interfaces
- **Orchestrator**: Main class for the orchestrator.
- **DataSubscriptionManager**: Manages subscriptions to multiple data streams with different refresh rates.
- **ModelInferenceCoordinator**: Coordinates inference across all models.
- **ModelOutputStore**: Stores and manages model outputs for cross-model feeding.
- **TrainingPipelineManager**: Manages training pipelines for all models.
- **DecisionMaker**: Interface for making trading decisions.
- **MoEGateway**: Mixture of Experts gateway for model integration.
#### Core Responsibilities
##### 1. Data Subscription and Management
The Orchestrator subscribes to the Data Provider and manages multiple data streams with varying refresh rates:
- **10Hz COB (Cumulative Order Book) Data**: High-frequency order book updates for real-time market depth analysis
- **OHLCV Data**: Traditional candlestick data at multiple timeframes (1s, 1m, 1h, 1d)
- **Market Tick Data**: Individual trade executions and price movements
- **Technical Indicators**: Calculated indicators that update at different frequencies
- **Pivot Points**: Market structure analysis data
**Data Stream Management**:
- Maintains separate buffers for each data type with appropriate retention policies
- Ensures thread-safe access to data streams from multiple models
- Implements intelligent caching to serve "last updated" data efficiently
- Maintains full base dataframe that stays current for any model requesting data
- Handles data synchronization across different refresh rates
**Enhanced 1s Timeseries Data Combination**:
- Combines OHLCV data with COB (Cumulative Order Book) data for 1s timeframes
- Implements price bucket aggregation: ±20 buckets around current price
- ETH: $1 bucket size (e.g., $3000-$3040 range = 40 buckets) when current price is 3020
- BTC: $10 bucket size (e.g., $50000-$50400 range = 40 buckets) when price is 50200
- Creates unified base data input that includes:
- Traditional OHLCV metrics (Open, High, Low, Close, Volume)
- Order book depth and liquidity at each price level
- Bid/ask imbalances for the +-5 buckets with Moving Averages for 5,15, and 60s
- Volume-weighted average prices within buckets
- Order flow dynamics and market microstructure data
##### 2. Model Inference Coordination
The Orchestrator coordinates inference across all models in the system:
**Inference Pipeline**:
- Triggers model inference when relevant data updates occur
- Manages inference scheduling based on data availability and model requirements
- Coordinates parallel inference execution for independent models
- Handles model dependencies (e.g., RL model waiting for CNN hidden states)
**Model Input Management**:
- Assembles appropriate input data for each model based on their requirements
- Ensures models receive the most current data available at inference time
- Manages feature engineering and data preprocessing for each model
- Handles different input formats and requirements across models
##### 3. Model Output Storage and Cross-Feeding
The Orchestrator maintains a centralized store for all model outputs and manages cross-model data feeding:
**Output Storage**:
- Stores CNN predictions, confidence scores, and hidden layer states
- Stores RL action recommendations and value estimates
- Stores outputs from all models in extensible format supporting future models (LSTM, Transformer, etc.)
- Maintains historical output sequences for temporal analysis
- Implements efficient retrieval mechanisms for real-time access
- Uses standardized ModelOutput format for easy extension and cross-model compatibility
**Cross-Model Feeding**:
- Feeds CNN hidden layer states into RL model inputs
- Provides CNN predictions as context for RL decision-making
- Includes "last predictions" from each available model as part of base data input
- Stores model outputs that become inputs for subsequent inference cycles
- Manages circular dependencies and feedback loops between models
- Supports dynamic model addition without requiring system architecture changes
##### 4. Training Pipeline Management
The Orchestrator coordinates training for all models by managing the prediction-result feedback loop:
**Training Coordination**:
- Calls each model's training pipeline when new inference results are available
- Provides previous predictions alongside new results for supervised learning
- Manages training data collection and labeling
- Coordinates online learning updates based on real-time performance
**Training Data Management**:
- Maintains training datasets with prediction-result pairs
- Implements data quality checks and filtering
- Manages training data retention and archival policies
- Provides training data statistics and monitoring
**Performance Tracking**:
- Tracks prediction accuracy for each model over time
- Monitors model performance degradation and triggers retraining
- Maintains performance metrics for model comparison and selection
**Training progress and checkpoints persistance**
- it uses the checkpoint manager to store check points of each model over time as training progresses and we have improvements
- checkpoint manager has capability to ensure only top 5 to 10 best checkpoints are stored for each model deleting the least performant ones. it stores metadata along the CPs to decide the performance
- we automatically load the best CP at startup if we have stored ones
##### 5. Inference Data Validation and Storage
The Orchestrator implements comprehensive inference data validation and persistent storage:
**Input Data Validation**:
- Validates complete OHLCV dataframes for all required timeframes before inference
- Checks input data dimensions against model requirements
- Logs missing components and prevents prediction on incomplete data
- Raises validation errors with specific details about expected vs actual dimensions
**Inference History Storage**:
- Stores complete input data packages with each prediction in persistent storage
- Includes timestamp, symbol, input features, prediction outputs, confidence scores, and model internal states
- Maintains compressed storage to minimize footprint while preserving accessibility
- Implements efficient query mechanisms by symbol, timeframe, and date range
**Storage Management**:
- Applies configurable retention policies to manage storage limits
- Archives or removes oldest entries when limits are reached
- Prioritizes keeping most recent and valuable training examples during storage pressure
- Provides data completeness metrics and validation results in logs
##### 6. Inference-Training Feedback Loop
The Orchestrator manages the continuous learning cycle through inference-training feedback:
**Prediction Outcome Evaluation**:
- Evaluates prediction accuracy against actual price movements after sufficient time has passed
- Creates training examples using stored inference data paired with actual market outcomes
- Feeds prediction-result pairs back to respective models for learning
**Adaptive Learning Signals**:
- Provides positive reinforcement signals for accurate predictions
- Delivers corrective training signals for inaccurate predictions to help models learn from mistakes
- Retrieves last inference data for each model to compare predictions against actual outcomes
**Continuous Improvement Tracking**:
- Tracks and reports accuracy improvements or degradations over time
- Monitors model learning progress through the feedback loop
- Alerts administrators when data flow issues are detected with specific error details and remediation suggestions
##### 5. Decision Making and Trading Actions
Beyond coordination, the Orchestrator makes final trading decisions:
**Decision Integration**:
- Combines outputs from CNN and RL models using Mixture of Experts approach
- Applies confidence-based filtering to avoid uncertain trades
- Implements configurable thresholds for buy/sell decisions
- Considers market conditions and risk parameters
#### Implementation Details
**Architecture**:
```python
class Orchestrator:
def __init__(self):
self.data_subscription_manager = DataSubscriptionManager()
self.model_inference_coordinator = ModelInferenceCoordinator()
self.model_output_store = ModelOutputStore()
self.training_pipeline_manager = TrainingPipelineManager()
self.decision_maker = DecisionMaker()
self.moe_gateway = MoEGateway()
async def run(self):
# Subscribe to data streams
await self.data_subscription_manager.subscribe_to_data_provider()
# Start inference coordination loop
await self.model_inference_coordinator.start()
# Start training pipeline management
await self.training_pipeline_manager.start()
```
**Data Flow Management**:
- Implements event-driven architecture for data updates
- Uses async/await patterns for non-blocking operations
- Maintains data freshness timestamps for each stream
- Implements backpressure handling for high-frequency data
**Model Coordination**:
- Manages model lifecycle (loading, inference, training, updating)
- Implements model versioning and rollback capabilities
- Handles model failures and fallback mechanisms
- Provides model performance monitoring and alerting
**Training Integration**:
- Implements incremental learning strategies
- Manages training batch composition and scheduling
- Provides training progress monitoring and control
- Handles training failures and recovery
### 5. Trading Executor
The Trading Executor is responsible for executing trading actions through brokerage APIs.
#### Key Classes and Interfaces
- **TradingExecutor**: Main class for the trading executor.
- **BrokerageAPI**: Interface for interacting with brokerages.
- **OrderManager**: Class for managing orders.
#### Implementation Details
The Trading Executor will:
- Accept trading actions from the orchestrator
- Execute orders through brokerage APIs
- Manage order lifecycle
- Handle errors and retries
- Provide feedback on order execution
Supported brokerages:
- MEXC
- Binance
- Bybit (future extension)
Order types:
- Market orders
- Limit orders
- Stop-loss orders
### 6. Risk Manager
The Risk Manager is responsible for implementing risk management features like stop-loss and position sizing.
#### Key Classes and Interfaces
- **RiskManager**: Main class for the risk manager.
- **StopLossManager**: Class for managing stop-loss orders.
- **PositionSizer**: Class for determining position sizes.
#### Implementation Details
The Risk Manager will:
- Implement configurable stop-loss functionality
- Implement configurable position sizing based on risk parameters
- Implement configurable maximum drawdown limits
- Provide real-time risk metrics
- Provide alerts for high-risk situations
Risk parameters:
- Maximum position size
- Maximum drawdown
- Risk per trade
- Maximum leverage
### 7. Dashboard
The Dashboard provides a user interface for monitoring and controlling the system.
#### Key Classes and Interfaces
- **Dashboard**: Main class for the dashboard.
- **ChartManager**: Class for managing charts.
- **ControlPanel**: Class for managing controls.
#### Implementation Details
The Dashboard will:
- Display real-time market data for all symbols and timeframes
- Display OHLCV charts for all timeframes
- Display CNN pivot point predictions and confidence levels
- Display RL and orchestrator trading actions and confidence levels
- Display system status and model performance metrics
- Provide start/stop toggles for all system processes
- Provide sliders to adjust buy/sell thresholds for the orchestrator
Implementation:
- Web-based dashboard using Flask/Dash
- Real-time updates using WebSockets
- Interactive charts using Plotly
- Server-side processing for all models
## Data Models
### Market Data
```python
@dataclass
class MarketTick:
symbol: str
timestamp: datetime
price: float
volume: float
quantity: float
side: str # 'buy' or 'sell'
trade_id: str
is_buyer_maker: bool
raw_data: Dict[str, Any] = field(default_factory=dict)
```
### OHLCV Data
```python
@dataclass
class OHLCVBar:
symbol: str
timestamp: datetime
open: float
high: float
low: float
close: float
volume: float
timeframe: str
indicators: Dict[str, float] = field(default_factory=dict)
```
### Pivot Points
```python
@dataclass
class PivotPoint:
symbol: str
timestamp: datetime
price: float
type: str # 'high' or 'low'
level: int # Pivot level (1, 2, 3, etc.)
confidence: float = 1.0
```
### Trading Actions
```python
@dataclass
class TradingAction:
symbol: str
timestamp: datetime
action: str # 'buy' or 'sell'
confidence: float
source: str # 'rl', 'cnn', 'orchestrator'
price: Optional[float] = None
quantity: Optional[float] = None
reason: Optional[str] = None
```
### Model Predictions
```python
@dataclass
class ModelOutput:
"""Extensible model output format supporting all model types"""
model_type: str # 'cnn', 'rl', 'lstm', 'transformer', 'orchestrator'
model_name: str # Specific model identifier
symbol: str
timestamp: datetime
confidence: float
predictions: Dict[str, Any] # Model-specific predictions
hidden_states: Optional[Dict[str, Any]] = None # For cross-model feeding
metadata: Dict[str, Any] = field(default_factory=dict) # Additional info
```
```python
@dataclass
class CNNPrediction:
symbol: str
timestamp: datetime
pivot_points: List[PivotPoint]
hidden_states: Dict[str, Any]
confidence: float
```
```python
@dataclass
class RLPrediction:
symbol: str
timestamp: datetime
action: str # 'buy' or 'sell'
confidence: float
expected_reward: float
```
### Enhanced Base Data Input
```python
@dataclass
class BaseDataInput:
"""Unified base data input for all models"""
symbol: str
timestamp: datetime
ohlcv_data: Dict[str, OHLCVBar] # Multi-timeframe OHLCV
cob_data: Optional[Dict[str, float]] = None # COB buckets for 1s timeframe
technical_indicators: Dict[str, float] = field(default_factory=dict)
pivot_points: List[PivotPoint] = field(default_factory=list)
last_predictions: Dict[str, ModelOutput] = field(default_factory=dict) # From all models
market_microstructure: Dict[str, Any] = field(default_factory=dict) # Order flow, etc.
```
### COB Data Structure
```python
@dataclass
class COBData:
"""Cumulative Order Book data for price buckets"""
symbol: str
timestamp: datetime
current_price: float
bucket_size: float # $1 for ETH, $10 for BTC
price_buckets: Dict[float, Dict[str, float]] # price -> {bid_volume, ask_volume, etc.}
bid_ask_imbalance: Dict[float, float] # price -> imbalance ratio
volume_weighted_prices: Dict[float, float] # price -> VWAP within bucket
order_flow_metrics: Dict[str, float] # Various order flow indicators
```
### Data Collection Errors
- Implement retry mechanisms for API failures
- Use fallback data sources when primary sources are unavailable
- Log all errors with detailed information
- Notify users through the dashboard
### Model Errors
- Implement model validation before deployment
- Use fallback models when primary models fail
- Log all errors with detailed information
- Notify users through the dashboard
### Trading Errors
- Implement order validation before submission
- Use retry mechanisms for order failures
- Implement circuit breakers for extreme market conditions
- Log all errors with detailed information
- Notify users through the dashboard
## Testing Strategy
### Unit Testing
- Test individual components in isolation
- Use mock objects for dependencies
- Focus on edge cases and error handling
### Integration Testing
- Test interactions between components
- Use real data for testing
- Focus on data flow and error propagation
### System Testing
- Test the entire system end-to-end
- Use real data for testing
- Focus on performance and reliability
### Backtesting
- Test trading strategies on historical data
- Measure performance metrics (PnL, Sharpe ratio, etc.)
- Compare against benchmarks
### Live Testing
- Test the system in a live environment with small position sizes
- Monitor performance and stability
- Gradually increase position sizes as confidence grows
## Implementation Plan
The implementation will follow a phased approach:
1. **Phase 1: Data Provider**
- Implement the enhanced data provider
- Implement pivot point calculation
- Implement technical indicator calculation
- Implement data normalization
2. **Phase 2: CNN Model**
- Implement the CNN model architecture
- Implement the training pipeline
- Implement the inference pipeline
- Implement the pivot point prediction
3. **Phase 3: RL Model**
- Implement the RL model architecture
- Implement the training pipeline
- Implement the inference pipeline
- Implement the trading action generation
4. **Phase 4: Orchestrator**
- Implement the orchestrator architecture
- Implement the decision-making logic
- Implement the MoE gateway
- Implement the confidence-based filtering
5. **Phase 5: Trading Executor**
- Implement the trading executor
- Implement the brokerage API integrations
- Implement the order management
- Implement the error handling
6. **Phase 6: Risk Manager**
- Implement the risk manager
- Implement the stop-loss functionality
- Implement the position sizing
- Implement the risk metrics
7. **Phase 7: Dashboard**
- Implement the dashboard UI
- Implement the chart management
- Implement the control panel
- Implement the real-time updates
8. **Phase 8: Integration and Testing**
- Integrate all components
- Implement comprehensive testing
- Fix bugs and optimize performance
- Deploy to production
## Monitoring and Visualization
### TensorBoard Integration (Future Enhancement)
A comprehensive TensorBoard integration has been designed to provide detailed training visualization and monitoring capabilities:
#### Features
- **Training Metrics Visualization**: Real-time tracking of model losses, rewards, and performance metrics
- **Feature Distribution Analysis**: Histograms and statistics of input features to validate data quality
- **State Quality Monitoring**: Tracking of comprehensive state building (13,400 features) success rates
- **Reward Component Analysis**: Detailed breakdown of reward calculations including PnL, confidence, volatility, and order flow
- **Model Performance Comparison**: Side-by-side comparison of CNN, RL, and orchestrator performance
#### Implementation Status
- **Completed**: TensorBoardLogger utility class with comprehensive logging methods
- **Completed**: Integration points in enhanced_rl_training_integration.py
- **Completed**: Enhanced run_tensorboard.py with improved visualization options
- **Status**: Ready for deployment when system stability is achieved
#### Usage
```bash
# Start TensorBoard dashboard
python run_tensorboard.py
# Access at http://localhost:6006
# View training metrics, feature distributions, and model performance
```
#### Benefits
- Real-time validation of training process
- Early detection of training issues
- Feature importance analysis
- Model performance comparison
- Historical training progress tracking
**Note**: TensorBoard integration is currently deprioritized in favor of system stability and core model improvements. It will be activated once the core training system is stable and performing optimally.
## Conclusion
This design document outlines the architecture, components, data flow, and implementation details for the Multi-Modal Trading System. The system is designed to be modular, extensible, and robust, with a focus on performance, reliability, and user experience.
The implementation will follow a phased approach, with each phase building on the previous one. The system will be thoroughly tested at each phase to ensure that it meets the requirements and performs as expected.
The final system will provide traders with a powerful tool for analyzing market data, identifying trading opportunities, and executing trades with confidence.
Reference in New Issue View Git Blame Copy Permalink