# Multi-Modal Trading System Design Document ## Overview The Multi-Modal Trading System is designed as an advanced algorithmic trading platform that combines Convolutional Neural Networks (CNN) and Reinforcement Learning (RL) models orchestrated by a decision-making module. The system processes multi-timeframe and multi-symbol market data (primarily ETH and BTC) to generate trading actions. This design document outlines the architecture, components, data flow, and implementation details for the system based on the requirements and existing codebase. ## Architecture The system follows a modular architecture with clear separation of concerns: ```mermaid graph TD A[Data Provider] --> B[Data Processor] (calculates pivot points) B --> C[CNN Model] B --> D[RL(DQN) Model] C --> E[Orchestrator] D --> E E --> F[Trading Executor] E --> G[Dashboard] F --> G H[Risk Manager] --> F H --> G ``` ### Key Components 1. **Data Provider**: Centralized component responsible for collecting, processing, and distributing market data from multiple sources. 2. **Data Processor**: Processes raw market data, calculates technical indicators, and identifies pivot points. 3. **CNN Model**: Analyzes patterns in market data and predicts pivot points across multiple timeframes. 4. **RL Model**: Learns optimal trading strategies based on market data and CNN predictions. 5. **Orchestrator**: Makes final trading decisions based on inputs from both CNN and RL models. 6. **Trading Executor**: Executes trading actions through brokerage APIs. 7. **Risk Manager**: Implements risk management features like stop-loss and position sizing. 8. **Dashboard**: Provides a user interface for monitoring and controlling the system. ## Components and Interfaces ### 1. Data Provider The Data Provider is the foundation of the system, responsible for collecting, processing, and distributing market data to all other components. #### Key Classes and Interfaces - **DataProvider**: Central class that manages data collection, processing, and distribution. - **MarketTick**: Data structure for standardized market tick data. - **DataSubscriber**: Interface for components that subscribe to market data. - **PivotBounds**: Data structure for pivot-based normalization bounds. #### Implementation Details The DataProvider class will: - Collect data from multiple sources (Binance, MEXC) - Support multiple timeframes (1s, 1m, 1h, 1d) - Support multiple symbols (ETH, BTC) - Calculate technical indicators - Identify pivot points - Normalize data - Distribute data to subscribers - Calculate any other algoritmic manipulations/calculations on the data - Cache up to 3x the model inputs (300 ticks OHLCV, etc) data so we can do a proper backtesting in up to 2x time in the future Based on the existing implementation in `core/data_provider.py`, we'll enhance it to: - Improve pivot point calculation using reccursive Williams Market Structure - Optimize data caching for better performance - Enhance real-time data streaming - Implement better error handling and fallback mechanisms ### BASE FOR ALL MODELS ### - ***INPUTS***: COB+OHCLV data frame as described: - OHCLV: 300 frames of (1s, 1m, 1h, 1d) ETH + 300s of 1s BTC - COB: for each 1s OHCLV we have +- 20 buckets of COB ammounts in USD - 1,5,15 and 60s MA of the COB imbalance counting +- 5 COB buckets - ***OUTPUTS***: suggested trade action (BUY/SELL) ### 2. CNN Model The CNN Model is responsible for analyzing patterns in market data and predicting pivot points across multiple timeframes. #### Key Classes and Interfaces - **CNNModel**: Main class for the CNN model. - **PivotPointPredictor**: Interface for predicting pivot points. - **CNNTrainer**: Class for training the CNN model. - ***INPUTS***: COB+OHCLV+Old Pivots (5 levels of pivots) - ***OUTPUTS***: next pivot point for each level as price-time vector. (can be plotted as trend line) + suggested trade action (BUY/SELL) #### Implementation Details The CNN Model will: - Accept multi-timeframe and multi-symbol data as input - Output predicted pivot points for each timeframe (1s, 1m, 1h, 1d) - Provide confidence scores for each prediction - Make hidden layer states available for the RL model Architecture: - Input layer: Multi-channel input for different timeframes and symbols - Convolutional layers: Extract patterns from time series data - LSTM/GRU layers: Capture temporal dependencies - Attention mechanism: Focus on relevant parts of the input - Output layer: Predict pivot points and confidence scores Training: - Use programmatically calculated pivot points as ground truth - Train on historical data - Update model when new pivot points are detected - Use backpropagation to optimize weights ### 3. RL Model The RL Model is responsible for learning optimal trading strategies based on market data and CNN predictions. #### Key Classes and Interfaces - **RLModel**: Main class for the RL model. - **TradingActionGenerator**: Interface for generating trading actions. - **RLTrainer**: Class for training the RL model. #### Implementation Details The RL Model will: - Accept market data, CNN model predictions (output), and CNN hidden layer states as input - Output trading action recommendations (buy/sell) - Provide confidence scores for each action - Learn from past experiences to adapt to the current market environment Architecture: - State representation: Market data, CNN model predictions (output), CNN hidden layer states - Action space: Buy, Sell - Reward function: PnL, risk-adjusted returns - Policy network: Deep neural network - Value network: Estimate expected returns Training: - Use reinforcement learning algorithms (DQN, PPO, A3C) - Train on historical data - Update model based on trading outcomes - Use experience replay to improve sample efficiency ### 4. Orchestrator The Orchestrator serves as the central coordination hub of the multi-modal trading system, responsible for data subscription management, model inference coordination, output storage, training pipeline orchestration, and inference-training feedback loop management. #### Key Classes and Interfaces - **Orchestrator**: Main class for the orchestrator. - **DataSubscriptionManager**: Manages subscriptions to multiple data streams with different refresh rates. - **ModelInferenceCoordinator**: Coordinates inference across all models. - **ModelOutputStore**: Stores and manages model outputs for cross-model feeding. - **TrainingPipelineManager**: Manages training pipelines for all models. - **DecisionMaker**: Interface for making trading decisions. - **MoEGateway**: Mixture of Experts gateway for model integration. #### Core Responsibilities ##### 1. Data Subscription and Management The Orchestrator subscribes to the Data Provider and manages multiple data streams with varying refresh rates: - **10Hz COB (Cumulative Order Book) Data**: High-frequency order book updates for real-time market depth analysis - **OHLCV Data**: Traditional candlestick data at multiple timeframes (1s, 1m, 1h, 1d) - **Market Tick Data**: Individual trade executions and price movements - **Technical Indicators**: Calculated indicators that update at different frequencies - **Pivot Points**: Market structure analysis data **Data Stream Management**: - Maintains separate buffers for each data type with appropriate retention policies - Ensures thread-safe access to data streams from multiple models - Implements intelligent caching to serve "last updated" data efficiently - Maintains full base dataframe that stays current for any model requesting data - Handles data synchronization across different refresh rates **Enhanced 1s Timeseries Data Combination**: - Combines OHLCV data with COB (Cumulative Order Book) data for 1s timeframes - Implements price bucket aggregation: ±20 buckets around current price - ETH: $1 bucket size (e.g., $3000-$3040 range = 40 buckets) when current price is 3020 - BTC: $10 bucket size (e.g., $50000-$50400 range = 40 buckets) when price is 50200 - Creates unified base data input that includes: - Traditional OHLCV metrics (Open, High, Low, Close, Volume) - Order book depth and liquidity at each price level - Bid/ask imbalances for the +-5 buckets with Moving Averages for 5,15, and 60s - Volume-weighted average prices within buckets - Order flow dynamics and market microstructure data ##### 2. Model Inference Coordination The Orchestrator coordinates inference across all models in the system: **Inference Pipeline**: - Triggers model inference when relevant data updates occur - Manages inference scheduling based on data availability and model requirements - Coordinates parallel inference execution for independent models - Handles model dependencies (e.g., RL model waiting for CNN hidden states) **Model Input Management**: - Assembles appropriate input data for each model based on their requirements - Ensures models receive the most current data available at inference time - Manages feature engineering and data preprocessing for each model - Handles different input formats and requirements across models ##### 3. Model Output Storage and Cross-Feeding The Orchestrator maintains a centralized store for all model outputs and manages cross-model data feeding: **Output Storage**: - Stores CNN predictions, confidence scores, and hidden layer states - Stores RL action recommendations and value estimates - Stores outputs from all models in extensible format supporting future models (LSTM, Transformer, etc.) - Maintains historical output sequences for temporal analysis - Implements efficient retrieval mechanisms for real-time access - Uses standardized ModelOutput format for easy extension and cross-model compatibility **Cross-Model Feeding**: - Feeds CNN hidden layer states into RL model inputs - Provides CNN predictions as context for RL decision-making - Includes "last predictions" from each available model as part of base data input - Stores model outputs that become inputs for subsequent inference cycles - Manages circular dependencies and feedback loops between models - Supports dynamic model addition without requiring system architecture changes ##### 4. Training Pipeline Management The Orchestrator coordinates training for all models by managing the prediction-result feedback loop: **Training Coordination**: - Calls each model's training pipeline when new inference results are available - Provides previous predictions alongside new results for supervised learning - Manages training data collection and labeling - Coordinates online learning updates based on real-time performance **Training Data Management**: - Maintains training datasets with prediction-result pairs - Implements data quality checks and filtering - Manages training data retention and archival policies - Provides training data statistics and monitoring **Performance Tracking**: - Tracks prediction accuracy for each model over time - Monitors model performance degradation and triggers retraining - Maintains performance metrics for model comparison and selection **Training progress and checkpoints persistance** - it uses the checkpoint manager to store check points of each model over time as training progresses and we have improvements - checkpoint manager has capability to ensure only top 5 to 10 best checkpoints are stored for each model deleting the least performant ones. it stores metadata along the CPs to decide the performance - we automatically load the best CP at startup if we have stored ones ##### 5. Inference Data Validation and Storage The Orchestrator implements comprehensive inference data validation and persistent storage: **Input Data Validation**: - Validates complete OHLCV dataframes for all required timeframes before inference - Checks input data dimensions against model requirements - Logs missing components and prevents prediction on incomplete data - Raises validation errors with specific details about expected vs actual dimensions **Inference History Storage**: - Stores complete input data packages with each prediction in persistent storage - Includes timestamp, symbol, input features, prediction outputs, confidence scores, and model internal states - Maintains compressed storage to minimize footprint while preserving accessibility - Implements efficient query mechanisms by symbol, timeframe, and date range **Storage Management**: - Applies configurable retention policies to manage storage limits - Archives or removes oldest entries when limits are reached - Prioritizes keeping most recent and valuable training examples during storage pressure - Provides data completeness metrics and validation results in logs ##### 6. Inference-Training Feedback Loop The Orchestrator manages the continuous learning cycle through inference-training feedback: **Prediction Outcome Evaluation**: - Evaluates prediction accuracy against actual price movements after sufficient time has passed - Creates training examples using stored inference data paired with actual market outcomes - Feeds prediction-result pairs back to respective models for learning **Adaptive Learning Signals**: - Provides positive reinforcement signals for accurate predictions - Delivers corrective training signals for inaccurate predictions to help models learn from mistakes - Retrieves last inference data for each model to compare predictions against actual outcomes **Continuous Improvement Tracking**: - Tracks and reports accuracy improvements or degradations over time - Monitors model learning progress through the feedback loop - Alerts administrators when data flow issues are detected with specific error details and remediation suggestions ##### 5. Decision Making and Trading Actions Beyond coordination, the Orchestrator makes final trading decisions: **Decision Integration**: - Combines outputs from CNN and RL models using Mixture of Experts approach - Applies confidence-based filtering to avoid uncertain trades - Implements configurable thresholds for buy/sell decisions - Considers market conditions and risk parameters #### Implementation Details **Architecture**: ```python class Orchestrator: def __init__(self): self.data_subscription_manager = DataSubscriptionManager() self.model_inference_coordinator = ModelInferenceCoordinator() self.model_output_store = ModelOutputStore() self.training_pipeline_manager = TrainingPipelineManager() self.decision_maker = DecisionMaker() self.moe_gateway = MoEGateway() async def run(self): # Subscribe to data streams await self.data_subscription_manager.subscribe_to_data_provider() # Start inference coordination loop await self.model_inference_coordinator.start() # Start training pipeline management await self.training_pipeline_manager.start() ``` **Data Flow Management**: - Implements event-driven architecture for data updates - Uses async/await patterns for non-blocking operations - Maintains data freshness timestamps for each stream - Implements backpressure handling for high-frequency data **Model Coordination**: - Manages model lifecycle (loading, inference, training, updating) - Implements model versioning and rollback capabilities - Handles model failures and fallback mechanisms - Provides model performance monitoring and alerting **Training Integration**: - Implements incremental learning strategies - Manages training batch composition and scheduling - Provides training progress monitoring and control - Handles training failures and recovery ### 5. Trading Executor The Trading Executor is responsible for executing trading actions through brokerage APIs. #### Key Classes and Interfaces - **TradingExecutor**: Main class for the trading executor. - **BrokerageAPI**: Interface for interacting with brokerages. - **OrderManager**: Class for managing orders. #### Implementation Details The Trading Executor will: - Accept trading actions from the orchestrator - Execute orders through brokerage APIs - Manage order lifecycle - Handle errors and retries - Provide feedback on order execution Supported brokerages: - MEXC - Binance - Bybit (future extension) Order types: - Market orders - Limit orders - Stop-loss orders ### 6. Risk Manager The Risk Manager is responsible for implementing risk management features like stop-loss and position sizing. #### Key Classes and Interfaces - **RiskManager**: Main class for the risk manager. - **StopLossManager**: Class for managing stop-loss orders. - **PositionSizer**: Class for determining position sizes. #### Implementation Details The Risk Manager will: - Implement configurable stop-loss functionality - Implement configurable position sizing based on risk parameters - Implement configurable maximum drawdown limits - Provide real-time risk metrics - Provide alerts for high-risk situations Risk parameters: - Maximum position size - Maximum drawdown - Risk per trade - Maximum leverage ### 7. Dashboard The Dashboard provides a user interface for monitoring and controlling the system. #### Key Classes and Interfaces - **Dashboard**: Main class for the dashboard. - **ChartManager**: Class for managing charts. - **ControlPanel**: Class for managing controls. #### Implementation Details The Dashboard will: - Display real-time market data for all symbols and timeframes - Display OHLCV charts for all timeframes - Display CNN pivot point predictions and confidence levels - Display RL and orchestrator trading actions and confidence levels - Display system status and model performance metrics - Provide start/stop toggles for all system processes - Provide sliders to adjust buy/sell thresholds for the orchestrator Implementation: - Web-based dashboard using Flask/Dash - Real-time updates using WebSockets - Interactive charts using Plotly - Server-side processing for all models ## Data Models ### Market Data ```python @dataclass class MarketTick: symbol: str timestamp: datetime price: float volume: float quantity: float side: str # 'buy' or 'sell' trade_id: str is_buyer_maker: bool raw_data: Dict[str, Any] = field(default_factory=dict) ``` ### OHLCV Data ```python @dataclass class OHLCVBar: symbol: str timestamp: datetime open: float high: float low: float close: float volume: float timeframe: str indicators: Dict[str, float] = field(default_factory=dict) ``` ### Pivot Points ```python @dataclass class PivotPoint: symbol: str timestamp: datetime price: float type: str # 'high' or 'low' level: int # Pivot level (1, 2, 3, etc.) confidence: float = 1.0 ``` ### Trading Actions ```python @dataclass class TradingAction: symbol: str timestamp: datetime action: str # 'buy' or 'sell' confidence: float source: str # 'rl', 'cnn', 'orchestrator' price: Optional[float] = None quantity: Optional[float] = None reason: Optional[str] = None ``` ### Model Predictions ```python @dataclass class ModelOutput: """Extensible model output format supporting all model types""" model_type: str # 'cnn', 'rl', 'lstm', 'transformer', 'orchestrator' model_name: str # Specific model identifier symbol: str timestamp: datetime confidence: float predictions: Dict[str, Any] # Model-specific predictions hidden_states: Optional[Dict[str, Any]] = None # For cross-model feeding metadata: Dict[str, Any] = field(default_factory=dict) # Additional info ``` ```python @dataclass class CNNPrediction: symbol: str timestamp: datetime pivot_points: List[PivotPoint] hidden_states: Dict[str, Any] confidence: float ``` ```python @dataclass class RLPrediction: symbol: str timestamp: datetime action: str # 'buy' or 'sell' confidence: float expected_reward: float ``` ### Enhanced Base Data Input ```python @dataclass class BaseDataInput: """Unified base data input for all models""" symbol: str timestamp: datetime ohlcv_data: Dict[str, OHLCVBar] # Multi-timeframe OHLCV cob_data: Optional[Dict[str, float]] = None # COB buckets for 1s timeframe technical_indicators: Dict[str, float] = field(default_factory=dict) pivot_points: List[PivotPoint] = field(default_factory=list) last_predictions: Dict[str, ModelOutput] = field(default_factory=dict) # From all models market_microstructure: Dict[str, Any] = field(default_factory=dict) # Order flow, etc. ``` ### COB Data Structure ```python @dataclass class COBData: """Cumulative Order Book data for price buckets""" symbol: str timestamp: datetime current_price: float bucket_size: float # $1 for ETH, $10 for BTC price_buckets: Dict[float, Dict[str, float]] # price -> {bid_volume, ask_volume, etc.} bid_ask_imbalance: Dict[float, float] # price -> imbalance ratio volume_weighted_prices: Dict[float, float] # price -> VWAP within bucket order_flow_metrics: Dict[str, float] # Various order flow indicators ``` ### Data Collection Errors - Implement retry mechanisms for API failures - Use fallback data sources when primary sources are unavailable - Log all errors with detailed information - Notify users through the dashboard ### Model Errors - Implement model validation before deployment - Use fallback models when primary models fail - Log all errors with detailed information - Notify users through the dashboard ### Trading Errors - Implement order validation before submission - Use retry mechanisms for order failures - Implement circuit breakers for extreme market conditions - Log all errors with detailed information - Notify users through the dashboard ## Testing Strategy ### Unit Testing - Test individual components in isolation - Use mock objects for dependencies - Focus on edge cases and error handling ### Integration Testing - Test interactions between components - Use real data for testing - Focus on data flow and error propagation ### System Testing - Test the entire system end-to-end - Use real data for testing - Focus on performance and reliability ### Backtesting - Test trading strategies on historical data - Measure performance metrics (PnL, Sharpe ratio, etc.) - Compare against benchmarks ### Live Testing - Test the system in a live environment with small position sizes - Monitor performance and stability - Gradually increase position sizes as confidence grows ## Implementation Plan The implementation will follow a phased approach: 1. **Phase 1: Data Provider** - Implement the enhanced data provider - Implement pivot point calculation - Implement technical indicator calculation - Implement data normalization 2. **Phase 2: CNN Model** - Implement the CNN model architecture - Implement the training pipeline - Implement the inference pipeline - Implement the pivot point prediction 3. **Phase 3: RL Model** - Implement the RL model architecture - Implement the training pipeline - Implement the inference pipeline - Implement the trading action generation 4. **Phase 4: Orchestrator** - Implement the orchestrator architecture - Implement the decision-making logic - Implement the MoE gateway - Implement the confidence-based filtering 5. **Phase 5: Trading Executor** - Implement the trading executor - Implement the brokerage API integrations - Implement the order management - Implement the error handling 6. **Phase 6: Risk Manager** - Implement the risk manager - Implement the stop-loss functionality - Implement the position sizing - Implement the risk metrics 7. **Phase 7: Dashboard** - Implement the dashboard UI - Implement the chart management - Implement the control panel - Implement the real-time updates 8. **Phase 8: Integration and Testing** - Integrate all components - Implement comprehensive testing - Fix bugs and optimize performance - Deploy to production ## Monitoring and Visualization ### TensorBoard Integration (Future Enhancement) A comprehensive TensorBoard integration has been designed to provide detailed training visualization and monitoring capabilities: #### Features - **Training Metrics Visualization**: Real-time tracking of model losses, rewards, and performance metrics - **Feature Distribution Analysis**: Histograms and statistics of input features to validate data quality - **State Quality Monitoring**: Tracking of comprehensive state building (13,400 features) success rates - **Reward Component Analysis**: Detailed breakdown of reward calculations including PnL, confidence, volatility, and order flow - **Model Performance Comparison**: Side-by-side comparison of CNN, RL, and orchestrator performance #### Implementation Status - **Completed**: TensorBoardLogger utility class with comprehensive logging methods - **Completed**: Integration points in enhanced_rl_training_integration.py - **Completed**: Enhanced run_tensorboard.py with improved visualization options - **Status**: Ready for deployment when system stability is achieved #### Usage ```bash # Start TensorBoard dashboard python run_tensorboard.py # Access at http://localhost:6006 # View training metrics, feature distributions, and model performance ``` #### Benefits - Real-time validation of training process - Early detection of training issues - Feature importance analysis - Model performance comparison - Historical training progress tracking **Note**: TensorBoard integration is currently deprioritized in favor of system stability and core model improvements. It will be activated once the core training system is stable and performing optimally. ## Conclusion This design document outlines the architecture, components, data flow, and implementation details for the Multi-Modal Trading System. The system is designed to be modular, extensible, and robust, with a focus on performance, reliability, and user experience. The implementation will follow a phased approach, with each phase building on the previous one. The system will be thoroughly tested at each phase to ensure that it meets the requirements and performs as expected. The final system will provide traders with a powerful tool for analyzing market data, identifying trading opportunities, and executing trades with confidence.