134 lines
4.6 KiB
Markdown
134 lines
4.6 KiB
Markdown
# COB System Architecture Analysis
|
|
|
|
## Overview
|
|
Analysis of the Consolidated Order Book (COB) system architecture, data sources, redundancies, and launch configurations.
|
|
|
|
## Questions & Answers
|
|
|
|
### 1. Do the COB dashboard and 1B model training use the same data source?
|
|
|
|
**Answer: YES** - but with redundant implementations.
|
|
|
|
**Data Flow:**
|
|
```
|
|
MultiExchangeCOBProvider (core/multi_exchange_cob_provider.py)
|
|
↓
|
|
COBIntegration (core/cob_integration.py)
|
|
↓
|
|
├── COB Dashboard (web/cob_realtime_dashboard.py)
|
|
├── Enhanced Orchestrator (core/enhanced_orchestrator.py)
|
|
└── RealtimeRLCOBTrader (core/realtime_rl_cob_trader.py)
|
|
```
|
|
|
|
**Current Implementation:**
|
|
- **Dashboard**: Creates own `COBIntegration(symbols=self.symbols)`
|
|
- **Training Pipeline**: Uses `EnhancedTradingOrchestrator` with deferred COB integration
|
|
- **1B RL Trader**: Creates own `COBIntegration(symbols=self.symbols)`
|
|
|
|
### 2. Are there redundant implementations?
|
|
|
|
**YES** - Significant redundancies identified:
|
|
|
|
#### Data Connection Redundancies:
|
|
- ✗ **Multiple WebSocket connections** to same exchanges (Binance, etc.)
|
|
- ✗ **Duplicate order book processing** across components
|
|
- ✗ **Multiple COBIntegration instances** instead of shared service
|
|
- ✗ **Redundant memory usage** for order book caches
|
|
|
|
#### Processing Redundancies:
|
|
- ✗ Same market data consolidated multiple times
|
|
- ✗ Duplicate price bucket calculations
|
|
- ✗ Multiple exchange breakdown computations
|
|
|
|
### 3. Combined Launch Script Status
|
|
|
|
**AVAILABLE** - Multiple options exist:
|
|
|
|
#### Existing Scripts:
|
|
1. **`run_integrated_rl_cob_dashboard.py`** ✅
|
|
- Combines RL trader + Dashboard in single process
|
|
- Integrated prediction display
|
|
- Shared COB data source (optimal)
|
|
|
|
2. **Separate Scripts** (redundant):
|
|
- `run_cob_dashboard.py`
|
|
- `run_realtime_rl_cob_trader.py`
|
|
- `run_enhanced_cob_training.py`
|
|
|
|
### 4. Launch Configuration Updates
|
|
|
|
**COMPLETED** - Added to `.vscode/launch.json`:
|
|
|
|
#### New Configurations:
|
|
1. **🚀 Integrated COB Dashboard + RL Trading**
|
|
- Single process with shared COB data
|
|
- Optimal resource usage
|
|
- Program: `run_integrated_rl_cob_dashboard.py`
|
|
|
|
2. **🔥 COB Dashboard + 1B RL Trading System** (Compound)
|
|
- Runs dashboard and RL trader separately
|
|
- Higher resource usage but better isolation
|
|
- Configurations: Dashboard + RL Trader
|
|
|
|
## Recommendations
|
|
|
|
### Immediate Actions:
|
|
1. **Use Integrated Script** - `run_integrated_rl_cob_dashboard.py` for optimal efficiency
|
|
2. **Launch via**: `🚀 Integrated COB Dashboard + RL Trading` configuration
|
|
|
|
### Architecture Improvements (Future):
|
|
1. **Shared COB Service** - Single COBIntegration instance as shared service
|
|
2. **Message Bus** - Distribute COB updates via event system
|
|
3. **Resource Pooling** - Share WebSocket connections and order book caches
|
|
|
|
## Usage Guide
|
|
|
|
### Launch Options (Ordered by Efficiency):
|
|
|
|
1. **🚀 Integrated COB Dashboard + RL Trading** (RECOMMENDED)
|
|
- Single process, shared resources
|
|
- Real-time RL predictions in dashboard
|
|
- Optimal memory usage
|
|
|
|
2. **🔥 COB Dashboard + 1B RL Trading System** (Compound)
|
|
- Separate processes for isolation
|
|
- Higher resource usage
|
|
- Better for debugging individual components
|
|
|
|
3. **Individual Scripts** (Development only)
|
|
- Separate dashboard or RL trader
|
|
- Highest resource usage
|
|
- Use only for component-specific debugging
|
|
|
|
## Technical Details
|
|
|
|
### COB Data Flow:
|
|
```
|
|
Exchange APIs → WebSocket Streams → MultiExchangeCOBProvider
|
|
↓
|
|
COBIntegration (callbacks & feature extraction)
|
|
↓
|
|
├── Dashboard (real-time visualization)
|
|
├── RL Models (1B parameter training)
|
|
└── Trading Executor (signal execution)
|
|
```
|
|
|
|
### Memory Usage Comparison:
|
|
- **Integrated**: ~4GB (shared COB data)
|
|
- **Compound**: ~6-8GB (duplicate COB instances)
|
|
- **Separate**: ~2-3GB each (multiple duplications)
|
|
|
|
### Current COB Features:
|
|
- ✅ Multi-exchange aggregation (Binance, Coinbase, Kraken, etc.)
|
|
- ✅ Real-time order book consolidation
|
|
- ✅ Fine-grain price buckets ($10 BTC, $1 ETH)
|
|
- ✅ CNN/DQN feature generation
|
|
- ✅ Session Volume Profile (SVP)
|
|
- ✅ Market microstructure analysis
|
|
- ✅ Dashboard integration with WebSocket streaming
|
|
|
|
## Conclusion
|
|
|
|
The COB system is well-architected with a solid data source (`MultiExchangeCOBProvider`), but current implementations create redundant instances. The **integrated script** (`run_integrated_rl_cob_dashboard.py`) already solves this by sharing COB data between dashboard and RL training, and has been added to launch configurations for easy access.
|
|
|
|
**Recommended Usage**: Use `🚀 Integrated COB Dashboard + RL Trading` launch configuration for optimal resource utilization and functionality. |