252 lines
8.7 KiB
Markdown
252 lines
8.7 KiB
Markdown
# COB RL Model Architecture Documentation
|
||
|
||
**Status**: REMOVED (Preserved for Future Recreation)
|
||
**Date**: 2025-01-03
|
||
**Reason**: Clean up code while preserving architecture for future improvement when quality COB data is available
|
||
|
||
## Overview
|
||
|
||
The COB (Consolidated Order Book) RL Model was a massive 356M+ parameter neural network specifically designed for real-time market microstructure analysis and trading decisions based on order book data.
|
||
|
||
## Architecture Details
|
||
|
||
### Core Network: `MassiveRLNetwork`
|
||
|
||
**Input**: 2000-dimensional COB features
|
||
**Target Parameters**: ~356M (optimized from initial 1B target)
|
||
**Inference Target**: 200ms cycles for ultra-low latency trading
|
||
|
||
#### Layer Structure:
|
||
|
||
```python
|
||
class MassiveRLNetwork(nn.Module):
|
||
def __init__(self, input_size=2000, hidden_size=2048, num_layers=8):
|
||
# Input projection layer
|
||
self.input_projection = nn.Sequential(
|
||
nn.Linear(input_size, hidden_size), # 2000 -> 2048
|
||
nn.LayerNorm(hidden_size),
|
||
nn.GELU(),
|
||
nn.Dropout(0.1)
|
||
)
|
||
|
||
# 8 Transformer encoder layers (main parameter bulk)
|
||
self.encoder_layers = nn.ModuleList([
|
||
nn.TransformerEncoderLayer(
|
||
d_model=2048, # Hidden dimension
|
||
nhead=16, # 16 attention heads
|
||
dim_feedforward=6144, # 3x hidden (6K feedforward)
|
||
dropout=0.1,
|
||
activation='gelu',
|
||
batch_first=True
|
||
) for _ in range(8) # 8 layers
|
||
])
|
||
|
||
# Market regime understanding
|
||
self.regime_encoder = nn.Sequential(
|
||
nn.Linear(2048, 2560), # Expansion layer
|
||
nn.LayerNorm(2560),
|
||
nn.GELU(),
|
||
nn.Dropout(0.1),
|
||
nn.Linear(2560, 2048), # Back to hidden size
|
||
nn.LayerNorm(2048),
|
||
nn.GELU()
|
||
)
|
||
|
||
# Output heads
|
||
self.price_head = ... # 3-class: DOWN/SIDEWAYS/UP
|
||
self.value_head = ... # RL value estimation
|
||
self.confidence_head = ... # Confidence [0,1]
|
||
```
|
||
|
||
#### Parameter Breakdown:
|
||
- **Input Projection**: ~4M parameters (2000×2048 + bias)
|
||
- **Transformer Layers**: ~320M parameters (8 layers × ~40M each)
|
||
- **Regime Encoder**: ~10M parameters
|
||
- **Output Heads**: ~15M parameters
|
||
- **Total**: ~356M parameters
|
||
|
||
### Model Interface: `COBRLModelInterface`
|
||
|
||
Wrapper class providing:
|
||
- Model management and lifecycle
|
||
- Training step functionality with mixed precision
|
||
- Checkpoint saving/loading
|
||
- Prediction interface
|
||
- Memory usage estimation
|
||
|
||
#### Key Features:
|
||
```python
|
||
class COBRLModelInterface(ModelInterface):
|
||
def __init__(self):
|
||
self.model = MassiveRLNetwork().to(device)
|
||
self.optimizer = torch.optim.AdamW(lr=1e-5, weight_decay=1e-6)
|
||
self.scaler = torch.cuda.amp.GradScaler() # Mixed precision
|
||
|
||
def predict(self, cob_features) -> Dict[str, Any]:
|
||
# Returns: predicted_direction, confidence, value, probabilities
|
||
|
||
def train_step(self, features, targets) -> float:
|
||
# Combined loss: direction + value + confidence
|
||
# Uses gradient clipping and mixed precision
|
||
```
|
||
|
||
## Input Data Format
|
||
|
||
### COB Features (2000-dimensional):
|
||
The model expected structured COB features containing:
|
||
- **Order Book Levels**: Bid/ask prices and volumes at multiple levels
|
||
- **Market Microstructure**: Spread, depth, imbalance ratios
|
||
- **Temporal Features**: Order flow dynamics, recent changes
|
||
- **Aggregated Metrics**: Volume-weighted averages, momentum indicators
|
||
|
||
### Target Training Data:
|
||
```python
|
||
targets = {
|
||
'direction': torch.tensor([0, 1, 2]), # 0=DOWN, 1=SIDEWAYS, 2=UP
|
||
'value': torch.tensor([reward_value]), # RL value estimation
|
||
'confidence': torch.tensor([0.0, 1.0]) # Confidence in prediction
|
||
}
|
||
```
|
||
|
||
## Training Methodology
|
||
|
||
### Loss Function:
|
||
```python
|
||
def _calculate_loss(outputs, targets):
|
||
direction_loss = F.cross_entropy(outputs['price_logits'], targets['direction'])
|
||
value_loss = F.mse_loss(outputs['value'], targets['value'])
|
||
confidence_loss = F.binary_cross_entropy(outputs['confidence'], targets['confidence'])
|
||
|
||
total_loss = direction_loss + 0.5 * value_loss + 0.3 * confidence_loss
|
||
return total_loss
|
||
```
|
||
|
||
### Optimization:
|
||
- **Optimizer**: AdamW with low learning rate (1e-5)
|
||
- **Weight Decay**: 1e-6 for regularization
|
||
- **Gradient Clipping**: Max norm 1.0
|
||
- **Mixed Precision**: CUDA AMP for efficiency
|
||
- **Batch Processing**: Designed for mini-batch training
|
||
|
||
## Integration Points
|
||
|
||
### In Trading Orchestrator:
|
||
```python
|
||
# Model initialization
|
||
self.cob_rl_agent = COBRLModelInterface()
|
||
|
||
# During prediction
|
||
cob_features = self._extract_cob_features(symbol) # 2000-dim array
|
||
prediction = self.cob_rl_agent.predict(cob_features)
|
||
```
|
||
|
||
### COB Data Flow:
|
||
```
|
||
COB Integration -> Feature Extraction -> MassiveRLNetwork -> Trading Decision
|
||
^ ^ ^ ^
|
||
COB Provider (2000 features) (356M params) (BUY/SELL/HOLD)
|
||
```
|
||
|
||
## Performance Characteristics
|
||
|
||
### Memory Usage:
|
||
- **Model Parameters**: ~1.4GB (356M × 4 bytes)
|
||
- **Activations**: ~100MB (during inference)
|
||
- **Total GPU Memory**: ~2GB for inference, ~4GB for training
|
||
|
||
### Computational Complexity:
|
||
- **FLOPs per Inference**: ~700M operations
|
||
- **Target Latency**: 200ms per prediction
|
||
- **Hardware Requirements**: GPU with 4GB+ VRAM
|
||
|
||
## Issues Identified
|
||
|
||
### Data Quality Problems:
|
||
1. **COB Data Inconsistency**: Raw COB data had quality issues
|
||
2. **Feature Engineering**: 2000-dimensional features needed better preprocessing
|
||
3. **Missing Market Context**: Isolated COB analysis without broader market view
|
||
4. **Temporal Alignment**: COB timestamps not properly synchronized
|
||
|
||
### Architecture Limitations:
|
||
1. **Massive Parameter Count**: 356M params for specialized task may be overkill
|
||
2. **Context Isolation**: No integration with price/volume patterns from other models
|
||
3. **Training Data**: Insufficient quality labeled data for RL training
|
||
4. **Real-time Performance**: 200ms latency target challenging for 356M model
|
||
|
||
## Future Improvement Strategy
|
||
|
||
### When COB Data Quality is Resolved:
|
||
|
||
#### Phase 1: Data Infrastructure
|
||
```python
|
||
# Improved COB data pipeline
|
||
class HighQualityCOBProvider:
|
||
def __init__(self):
|
||
self.quality_validators = [...]
|
||
self.feature_normalizers = [...]
|
||
self.temporal_aligners = [...]
|
||
|
||
def get_quality_cob_features(self, symbol: str) -> np.ndarray:
|
||
# Return validated, normalized, properly timestamped COB features
|
||
pass
|
||
```
|
||
|
||
#### Phase 2: Architecture Optimization
|
||
```python
|
||
# More efficient architecture
|
||
class OptimizedCOBNetwork(nn.Module):
|
||
def __init__(self, input_size=1000, hidden_size=1024, num_layers=6):
|
||
# Reduced parameter count: ~100M instead of 356M
|
||
# Better efficiency while maintaining capability
|
||
pass
|
||
```
|
||
|
||
#### Phase 3: Integration Enhancement
|
||
```python
|
||
# Hybrid approach: COB + Market Context
|
||
class HybridCOBCNNModel(nn.Module):
|
||
def __init__(self):
|
||
self.cob_encoder = OptimizedCOBNetwork()
|
||
self.market_encoder = EnhancedCNN()
|
||
self.fusion_layer = AttentionFusion()
|
||
|
||
def forward(self, cob_features, market_features):
|
||
# Combine COB microstructure with broader market patterns
|
||
pass
|
||
```
|
||
|
||
## Removal Justification
|
||
|
||
### Why Removed Now:
|
||
1. **COB Data Quality**: Current COB data pipeline has quality issues
|
||
2. **Parameter Efficiency**: 356M params not justified without quality data
|
||
3. **Development Focus**: Better to fix data pipeline first
|
||
4. **Code Cleanliness**: Remove complexity while preserving knowledge
|
||
|
||
### Preservation Strategy:
|
||
1. **Complete Documentation**: This document preserves full architecture
|
||
2. **Interface Compatibility**: Easy to recreate interface when needed
|
||
3. **Test Framework**: Existing tests can validate future recreation
|
||
4. **Integration Points**: Clear documentation of how to reintegrate
|
||
|
||
## Recreation Checklist
|
||
|
||
When ready to recreate an improved COB model:
|
||
|
||
- [ ] Verify COB data quality and consistency
|
||
- [ ] Implement proper feature engineering pipeline
|
||
- [ ] Design architecture with appropriate parameter count
|
||
- [ ] Create comprehensive training dataset
|
||
- [ ] Implement proper integration with other models
|
||
- [ ] Validate real-time performance requirements
|
||
- [ ] Test extensively before production deployment
|
||
|
||
## Code Preservation
|
||
|
||
Original files preserved in git history:
|
||
- `NN/models/cob_rl_model.py` (full implementation)
|
||
- Integration code in `core/orchestrator.py`
|
||
- Related test files
|
||
|
||
**Note**: This documentation ensures the COB model can be accurately recreated when COB data quality issues are resolved and the massive parameter advantage can be properly evaluated.
|