popov/gogo2
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

kiro design and requirements

Browse Source
This commit is contained in:
Dobromir Popov
2025-07-17 00:57:50 +03:00
parent 8023dae18f
commit f73cd17dfc
3 changed files with 571 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

439
.kiro/specs/multi-modal-trading-system/design.md Normal file
View File

@ -0,0 +1,439 @@
# Multi-Modal Trading System Design Document
## Overview
The Multi-Modal Trading System is designed as an advanced algorithmic trading platform that combines Convolutional Neural Networks (CNN) and Reinforcement Learning (RL) models orchestrated by a decision-making module. The system processes multi-timeframe and multi-symbol market data (primarily ETH and BTC) to generate trading actions.
This design document outlines the architecture, components, data flow, and implementation details for the system based on the requirements and existing codebase.
## Architecture
The system follows a modular architecture with clear separation of concerns:
```mermaid
graph TD
A[Data Provider] --> B[Data Processor]
B --> C[CNN Model]
B --> D[RL Model]
C --> E[Orchestrator]
D --> E
E --> F[Trading Executor]
E --> G[Dashboard]
F --> G
H[Risk Manager] --> F
H --> G
```
### Key Components
1. **Data Provider**: Centralized component responsible for collecting, processing, and distributing market data from multiple sources.
2. **Data Processor**: Processes raw market data, calculates technical indicators, and identifies pivot points.
3. **CNN Model**: Analyzes patterns in market data and predicts pivot points across multiple timeframes.
4. **RL Model**: Learns optimal trading strategies based on market data and CNN predictions.
5. **Orchestrator**: Makes final trading decisions based on inputs from both CNN and RL models.
6. **Trading Executor**: Executes trading actions through brokerage APIs.
7. **Risk Manager**: Implements risk management features like stop-loss and position sizing.
8. **Dashboard**: Provides a user interface for monitoring and controlling the system.
## Components and Interfaces
### 1. Data Provider
The Data Provider is the foundation of the system, responsible for collecting, processing, and distributing market data to all other components.
#### Key Classes and Interfaces
- **DataProvider**: Central class that manages data collection, processing, and distribution.
- **MarketTick**: Data structure for standardized market tick data.
- **DataSubscriber**: Interface for components that subscribe to market data.
- **PivotBounds**: Data structure for pivot-based normalization bounds.
#### Implementation Details
The DataProvider class will:
- Collect data from multiple sources (Binance, MEXC)
- Support multiple timeframes (1s, 1m, 1h, 1d)
- Support multiple symbols (ETH, BTC)
- Calculate technical indicators
- Identify pivot points
- Normalize data
- Distribute data to subscribers
Based on the existing implementation in `core/data_provider.py`, we'll enhance it to:
- Improve pivot point calculation using Williams Market Structure
- Optimize data caching for better performance
- Enhance real-time data streaming
- Implement better error handling and fallback mechanisms
### 2. CNN Model
The CNN Model is responsible for analyzing patterns in market data and predicting pivot points across multiple timeframes.
#### Key Classes and Interfaces
- **CNNModel**: Main class for the CNN model.
- **PivotPointPredictor**: Interface for predicting pivot points.
- **CNNTrainer**: Class for training the CNN model.
#### Implementation Details
The CNN Model will:
- Accept multi-timeframe and multi-symbol data as input
- Output predicted pivot points for each timeframe (1s, 1m, 1h, 1d)
- Provide confidence scores for each prediction
- Make hidden layer states available for the RL model
Architecture:
- Input layer: Multi-channel input for different timeframes and symbols
- Convolutional layers: Extract patterns from time series data
- LSTM/GRU layers: Capture temporal dependencies
- Attention mechanism: Focus on relevant parts of the input
- Output layer: Predict pivot points and confidence scores
Training:
- Use programmatically calculated pivot points as ground truth
- Train on historical data
- Update model when new pivot points are detected
- Use backpropagation to optimize weights
### 3. RL Model
The RL Model is responsible for learning optimal trading strategies based on market data and CNN predictions.
#### Key Classes and Interfaces
- **RLModel**: Main class for the RL model.
- **TradingActionGenerator**: Interface for generating trading actions.
- **RLTrainer**: Class for training the RL model.
#### Implementation Details
The RL Model will:
- Accept market data, CNN predictions, and CNN hidden layer states as input
- Output trading action recommendations (buy/sell)
- Provide confidence scores for each action
- Learn from past experiences to adapt to the current market environment
Architecture:
- State representation: Market data, CNN predictions, CNN hidden layer states
- Action space: Buy, Sell
- Reward function: PnL, risk-adjusted returns
- Policy network: Deep neural network
- Value network: Estimate expected returns
Training:
- Use reinforcement learning algorithms (DQN, PPO, A3C)
- Train on historical data
- Update model based on trading outcomes
- Use experience replay to improve sample efficiency
### 4. Orchestrator
The Orchestrator is responsible for making final trading decisions based on inputs from both CNN and RL models.
#### Key Classes and Interfaces
- **Orchestrator**: Main class for the orchestrator.
- **DecisionMaker**: Interface for making trading decisions.
- **MoEGateway**: Mixture of Experts gateway for model integration.
#### Implementation Details
The Orchestrator will:
- Accept inputs from both CNN and RL models
- Output final trading actions (buy/sell)
- Consider confidence levels of both models
- Learn to avoid entering positions when uncertain
- Allow for configurable thresholds for entering and exiting positions
Architecture:
- Mixture of Experts (MoE) approach
- Gating network: Determine which expert to trust
- Expert models: CNN, RL, and potentially others
- Decision network: Combine expert outputs
Training:
- Train on historical data
- Update model based on trading outcomes
- Use reinforcement learning to optimize decision-making
### 5. Trading Executor
The Trading Executor is responsible for executing trading actions through brokerage APIs.
#### Key Classes and Interfaces
- **TradingExecutor**: Main class for the trading executor.
- **BrokerageAPI**: Interface for interacting with brokerages.
- **OrderManager**: Class for managing orders.
#### Implementation Details
The Trading Executor will:
- Accept trading actions from the orchestrator
- Execute orders through brokerage APIs
- Manage order lifecycle
- Handle errors and retries
- Provide feedback on order execution
Supported brokerages:
- MEXC
- Binance
- Bybit (future extension)
Order types:
- Market orders
- Limit orders
- Stop-loss orders
### 6. Risk Manager
The Risk Manager is responsible for implementing risk management features like stop-loss and position sizing.
#### Key Classes and Interfaces
- **RiskManager**: Main class for the risk manager.
- **StopLossManager**: Class for managing stop-loss orders.
- **PositionSizer**: Class for determining position sizes.
#### Implementation Details
The Risk Manager will:
- Implement configurable stop-loss functionality
- Implement configurable position sizing based on risk parameters
- Implement configurable maximum drawdown limits
- Provide real-time risk metrics
- Provide alerts for high-risk situations
Risk parameters:
- Maximum position size
- Maximum drawdown
- Risk per trade
- Maximum leverage
### 7. Dashboard
The Dashboard provides a user interface for monitoring and controlling the system.
#### Key Classes and Interfaces
- **Dashboard**: Main class for the dashboard.
- **ChartManager**: Class for managing charts.
- **ControlPanel**: Class for managing controls.
#### Implementation Details
The Dashboard will:
- Display real-time market data for all symbols and timeframes
- Display OHLCV charts for all timeframes
- Display CNN pivot point predictions and confidence levels
- Display RL and orchestrator trading actions and confidence levels
- Display system status and model performance metrics
- Provide start/stop toggles for all system processes
- Provide sliders to adjust buy/sell thresholds for the orchestrator
Implementation:
- Web-based dashboard using Flask/Dash
- Real-time updates using WebSockets
- Interactive charts using Plotly
- Server-side processing for all models
## Data Models
### Market Data
```python
@dataclass
class MarketTick:
symbol: str
timestamp: datetime
price: float
volume: float
quantity: float
side: str # 'buy' or 'sell'
trade_id: str
is_buyer_maker: bool
raw_data: Dict[str, Any] = field(default_factory=dict)
```
### OHLCV Data
```python
@dataclass
class OHLCVBar:
symbol: str
timestamp: datetime
open: float
high: float
low: float
close: float
volume: float
timeframe: str
indicators: Dict[str, float] = field(default_factory=dict)
```
### Pivot Points
```python
@dataclass
class PivotPoint:
symbol: str
timestamp: datetime
price: float
type: str # 'high' or 'low'
level: int # Pivot level (1, 2, 3, etc.)
confidence: float = 1.0
```
### Trading Actions
```python
@dataclass
class TradingAction:
symbol: str
timestamp: datetime
action: str # 'buy' or 'sell'
confidence: float
source: str # 'rl', 'cnn', 'orchestrator'
price: Optional[float] = None
quantity: Optional[float] = None
reason: Optional[str] = None
```
### Model Predictions
```python
@dataclass
class CNNPrediction:
symbol: str
timestamp: datetime
pivot_points: List[PivotPoint]
hidden_states: Dict[str, Any]
confidence: float
```
```python
@dataclass
class RLPrediction:
symbol: str
timestamp: datetime
action: str # 'buy' or 'sell'
confidence: float
expected_reward: float
```
## Error Handling
### Data Collection Errors
- Implement retry mechanisms for API failures
- Use fallback data sources when primary sources are unavailable
- Log all errors with detailed information
- Notify users through the dashboard
### Model Errors
- Implement model validation before deployment
- Use fallback models when primary models fail
- Log all errors with detailed information
- Notify users through the dashboard
### Trading Errors
- Implement order validation before submission
- Use retry mechanisms for order failures
- Implement circuit breakers for extreme market conditions
- Log all errors with detailed information
- Notify users through the dashboard
## Testing Strategy
### Unit Testing
- Test individual components in isolation
- Use mock objects for dependencies
- Focus on edge cases and error handling
### Integration Testing
- Test interactions between components
- Use real data for testing
- Focus on data flow and error propagation
### System Testing
- Test the entire system end-to-end
- Use real data for testing
- Focus on performance and reliability
### Backtesting
- Test trading strategies on historical data
- Measure performance metrics (PnL, Sharpe ratio, etc.)
- Compare against benchmarks
### Live Testing
- Test the system in a live environment with small position sizes
- Monitor performance and stability
- Gradually increase position sizes as confidence grows
## Implementation Plan
The implementation will follow a phased approach:
1. **Phase 1: Data Provider**
- Implement the enhanced data provider
- Implement pivot point calculation
- Implement technical indicator calculation
- Implement data normalization
2. **Phase 2: CNN Model**
- Implement the CNN model architecture
- Implement the training pipeline
- Implement the inference pipeline
- Implement the pivot point prediction
3. **Phase 3: RL Model**
- Implement the RL model architecture
- Implement the training pipeline
- Implement the inference pipeline
- Implement the trading action generation
4. **Phase 4: Orchestrator**
- Implement the orchestrator architecture
- Implement the decision-making logic
- Implement the MoE gateway
- Implement the confidence-based filtering
5. **Phase 5: Trading Executor**
- Implement the trading executor
- Implement the brokerage API integrations
- Implement the order management
- Implement the error handling
6. **Phase 6: Risk Manager**
- Implement the risk manager
- Implement the stop-loss functionality
- Implement the position sizing
- Implement the risk metrics
7. **Phase 7: Dashboard**
- Implement the dashboard UI
- Implement the chart management
- Implement the control panel
- Implement the real-time updates
8. **Phase 8: Integration and Testing**
- Integrate all components
- Implement comprehensive testing
- Fix bugs and optimize performance
- Deploy to production
## Conclusion
This design document outlines the architecture, components, data flow, and implementation details for the Multi-Modal Trading System. The system is designed to be modular, extensible, and robust, with a focus on performance, reliability, and user experience.
The implementation will follow a phased approach, with each phase building on the previous one. The system will be thoroughly tested at each phase to ensure that it meets the requirements and performs as expected.
The final system will provide traders with a powerful tool for analyzing market data, identifying trading opportunities, and executing trades with confidence.

132
.kiro/specs/multi-modal-trading-system/requirements.md Normal file
View File

@ -0,0 +1,132 @@
# Requirements Document
## Introduction
The Multi-Modal Trading System is an advanced algorithmic trading platform that combines Convolutional Neural Networks (CNN) and Reinforcement Learning (RL) models orchestrated by a decision-making module. The system processes multi-timeframe and multi-symbol market data (primarily ETH and BTC) to generate trading actions. The system is designed to adapt to current market conditions through continuous learning from past experiences, with the CNN module trained on historical data to predict pivot points and the RL module optimizing trading decisions based on these predictions and market data.
## Requirements
### Requirement 1: Data Collection and Processing
**User Story:** As a trader, I want the system to collect and process multi-timeframe and multi-symbol market data, so that the models have comprehensive market information for making accurate trading decisions.
#### Acceptance Criteria
1. WHEN the system starts THEN it SHALL collect and process data for both ETH and BTC symbols.
2. WHEN collecting data THEN the system SHALL store the following for the primary symbol (ETH):
- 300 seconds of raw tick data - price and COB snapshot for all prices +- 1% on fine reslolution buckets (1$ for ETH, 10$ for BTC)
- 300 seconds of 1-second OHLCV data + 1s aggregated COB data
- 300 bars of OHLCV + indicators for each timeframe (1s, 1m, 1h, 1d)
3. WHEN collecting data THEN the system SHALL store similar data for the reference symbol (BTC).
4. WHEN processing data THEN the system SHALL calculate standard technical indicators for all timeframes.
5. WHEN processing data THEN the system SHALL calculate pivot points for all timeframes according to the specified methodology.
6. WHEN new data arrives THEN the system SHALL update its data cache in real-time.
7. IF tick data is not available THEN the system SHALL substitute with the lowest available timeframe data.
8. WHEN normalizing data THEN the system SHALL normalize to the max and min of the highest timeframe to maintain relationships between different timeframes.
9. data is cached for longer (let's start with double the model inputs so 600 bars) to support performing backtesting when we know the current predictions outcomes so we can generate test cases.
10. In general all models have access to the whole data we collect in a central data provider implementation. only some are specialized.
### Requirement 2: CNN Model Implementation
**User Story:** As a trader, I want the system to implement a CNN model that can identify patterns and predict pivot points across multiple timeframes, so that I can anticipate market direction changes.
#### Acceptance Criteria
1. WHEN the CNN model is initialized THEN it SHALL accept multi-timeframe and multi-symbol data as input.
2. WHEN processing input data THEN the CNN model SHALL output predicted pivot points for each timeframe (1s, 1m, 1h, 1d).
3. WHEN predicting pivot points THEN the CNN model SHALL provide both the predicted pivot point value and the timestamp when it is expected to occur.
4. WHEN a pivot point is detected THEN the system SHALL trigger a training round for the CNN model using historical data.
5. WHEN training the CNN model THEN the system SHALL use programmatically calculated pivot points from historical data as ground truth.
6. WHEN outputting predictions THEN the CNN model SHALL include a confidence score for each prediction.
7. WHEN calculating pivot points THEN the system SHALL implement both standard pivot points and the recursive Williams market structure pivot points as described.
8. WHEN processing data THEN the CNN model SHALL make available its hidden layer states for use by the RL model.
### Requirement 3: RL Model Implementation
**User Story:** As a trader, I want the system to implement an RL model that can learn optimal trading strategies based on market data and CNN predictions, so that the system can adapt to changing market conditions.
#### Acceptance Criteria
1. WHEN the RL model is initialized THEN it SHALL accept market data, CNN predictions, and CNN hidden layer states as input.
2. WHEN processing input data THEN the RL model SHALL output trading action recommendations (buy/sell).
3. WHEN evaluating trading actions THEN the RL model SHALL learn from past experiences to adapt to the current market environment.
4. WHEN making decisions THEN the RL model SHALL consider the confidence levels of CNN predictions.
5. WHEN uncertain about market direction THEN the RL model SHALL learn to avoid entering positions.
6. WHEN training the RL model THEN the system SHALL use a reward function that incentivizes high risk/reward setups.
7. WHEN outputting trading actions THEN the RL model SHALL provide a confidence score for each action.
8. WHEN a trading action is executed THEN the system SHALL store the input data for future training.
### Requirement 4: Orchestrator Implementation
**User Story:** As a trader, I want the system to implement an orchestrator that can make final trading decisions based on inputs from both CNN and RL models, so that the system can make more balanced and informed trading decisions.
#### Acceptance Criteria
1. WHEN the orchestrator is initialized THEN it SHALL accept inputs from both CNN and RL models.
2. WHEN processing model inputs THEN the orchestrator SHALL output final trading actions (buy/sell).
3. WHEN making decisions THEN the orchestrator SHALL consider the confidence levels of both CNN and RL models.
4. WHEN uncertain about market direction THEN the orchestrator SHALL learn to avoid entering positions.
5. WHEN implementing the orchestrator THEN the system SHALL use a Mixture of Experts (MoE) approach to allow for future model integration.
6. WHEN outputting trading actions THEN the orchestrator SHALL provide a confidence score for each action.
7. WHEN a trading action is executed THEN the system SHALL store the input data for future training.
8. WHEN implementing the orchestrator THEN the system SHALL allow for configurable thresholds for entering and exiting positions.
### Requirement 5: Training Pipeline
**User Story:** As a developer, I want the system to implement a unified training pipeline for both CNN and RL models, so that the models can be trained efficiently and consistently.
#### Acceptance Criteria
1. WHEN training models THEN the system SHALL use a unified data provider to prepare data for all models.
2. WHEN a pivot point is detected THEN the system SHALL trigger a training round for the CNN model.
3. WHEN training the CNN model THEN the system SHALL use programmatically calculated pivot points from historical data as ground truth.
4. WHEN training the RL model THEN the system SHALL use a reward function that incentivizes high risk/reward setups.
5. WHEN training models THEN the system SHALL run the training process on the server without requiring the dashboard to be open.
6. WHEN training models THEN the system SHALL provide real-time feedback on training progress through the dashboard.
7. WHEN training models THEN the system SHALL store model checkpoints for future use.
8. WHEN training models THEN the system SHALL provide metrics on model performance.
### Requirement 6: Dashboard Implementation
**User Story:** As a trader, I want the system to implement a comprehensive dashboard that displays real-time data, model predictions, and trading actions, so that I can monitor the system's performance and make informed decisions.
#### Acceptance Criteria
1. WHEN the dashboard is initialized THEN it SHALL display real-time market data for all symbols and timeframes.
2. WHEN displaying market data THEN the dashboard SHALL show OHLCV charts for all timeframes.
3. WHEN displaying model predictions THEN the dashboard SHALL show CNN pivot point predictions and confidence levels.
4. WHEN displaying trading actions THEN the dashboard SHALL show RL and orchestrator trading actions and confidence levels.
5. WHEN displaying system status THEN the dashboard SHALL show training progress and model performance metrics.
6. WHEN implementing controls THEN the dashboard SHALL provide start/stop toggles for all system processes.
7. WHEN implementing controls THEN the dashboard SHALL provide sliders to adjust buy/sell thresholds for the orchestrator.
8. WHEN implementing the dashboard THEN the system SHALL ensure all processes run on the server without requiring the dashboard to be open.
### Requirement 7: Risk Management
**User Story:** As a trader, I want the system to implement risk management features, so that I can protect my capital from significant losses.
#### Acceptance Criteria
1. WHEN implementing risk management THEN the system SHALL provide configurable stop-loss functionality.
2. WHEN a stop-loss is triggered THEN the system SHALL automatically close the position.
3. WHEN implementing risk management THEN the system SHALL provide configurable position sizing based on risk parameters.
4. WHEN implementing risk management THEN the system SHALL provide configurable maximum drawdown limits.
5. WHEN maximum drawdown limits are reached THEN the system SHALL automatically stop trading.
6. WHEN implementing risk management THEN the system SHALL provide real-time risk metrics through the dashboard.
7. WHEN implementing risk management THEN the system SHALL allow for different risk parameters for different market conditions.
8. WHEN implementing risk management THEN the system SHALL provide alerts for high-risk situations.
### Requirement 8: System Architecture and Integration
**User Story:** As a developer, I want the system to implement a clean and modular architecture, so that the system is easy to maintain and extend.
#### Acceptance Criteria
1. WHEN implementing the system architecture THEN the system SHALL use a unified data provider to prepare data for all models.
2. WHEN implementing the system architecture THEN the system SHALL use a modular approach to allow for easy extension.
3. WHEN implementing the system architecture THEN the system SHALL use a clean separation of concerns between data collection, model training, and trading execution.
4. WHEN implementing the system architecture THEN the system SHALL use a unified interface for all models.
5. WHEN implementing the system architecture THEN the system SHALL use a unified interface for all data providers.
6. WHEN implementing the system architecture THEN the system SHALL use a unified interface for all trading executors.
7. WHEN implementing the system architecture THEN the system SHALL use a unified interface for all risk management components.
8. WHEN implementing the system architecture THEN the system SHALL use a unified interface for all dashboard components.

0
.kiro/specs/multi-modal-trading-system/tasks.md Normal file
View File